-- Logs begin at Tue 2022-09-06 00:32:21 UTC, end at Tue 2022-09-06 01:36:21 UTC. -- Sep 06 00:57:38.005784 dv-cinder-847732-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 00:57:38.016733 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 00:57:38 2022] *** Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: nodename: dv-cinder-847732-2-iscsi Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: machine: x86_64 Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: clock source: unix Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: pcre jit disabled Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: detected number of CPU cores: 4 Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: current working directory: / Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: detected binary path: /usr/bin/uwsgi-core Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: your processes number limit is 47680 Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: your memory page size is 4096 bytes Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: detected max file descriptor number: 2048 Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: lock engine: pthread robust mutexes Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: thunder lock: enabled Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 00:57:38.019848 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 00:57:38.057791 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: Python main interpreter initialized at 0x557827b769c0 Sep 06 00:57:38.057791 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: python threads support enabled Sep 06 00:57:38.057791 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: your server socket listen backlog is limited to 100 connections Sep 06 00:57:38.057791 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: your mercy for graceful operations on workers is 90 seconds Sep 06 00:57:38.058127 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 00:57:38.058206 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: *** Operational MODE: preforking *** Sep 06 00:57:38.058475 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: *** uWSGI is running in multiple interpreter mode *** Sep 06 00:57:38.058475 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: spawned uWSGI master process (pid: 81600) Sep 06 00:57:38.058669 dv-cinder-847732-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 00:57:38.060519 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: spawned uWSGI worker 1 (pid: 81610, cores: 1) Sep 06 00:57:38.060519 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: spawned uWSGI worker 2 (pid: 81611, cores: 1) Sep 06 00:57:38.060519 dv-cinder-847732-2-iscsi devstack@keystone.service[81600]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 00:57:42.337681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 00:57:42.337681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: from cryptography.utils import int_from_bytes Sep 06 00:57:42.338489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 00:57:42.338489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: from cryptography.utils import int_from_bytes Sep 06 00:57:42.341852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 00:57:42.341852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Sep 06 00:57:42.342360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 00:57:42.342360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Sep 06 00:57:42.388045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 00:57:42.388233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 00:57:42.388424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] command line args: [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 00:57:42.388692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 00:57:42.388870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ================================================================================ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 00:57:42.389124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] admin_token = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.389368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config_dir = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.389554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.389734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config_source = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.389912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] debug = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.390110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.390410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 00:57:42.390501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.390563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 00:57:42.390671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.390769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] command line args: [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 00:57:42.390890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.391033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 00:57:42.391146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.391254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ================================================================================ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 00:57:42.391349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.391448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.391546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_config_append = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.391690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_dir = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.391826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.391934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_dir = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_source = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] debug = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.392897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_config_append = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.393976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_dir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.394967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.395996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] publish_errors = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.396949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.397127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.397208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.397357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] standard_threads = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.397435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.397587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.397667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.397829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_journal = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] publish_errors = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_json = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_stderr = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_syslog = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.398973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.399139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.399221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.399357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.399437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.399595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] standard_threads = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.399679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.399852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.399932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.400099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.400188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.400260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.400329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.400438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_journal = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.400567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.400651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_json = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.400775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_stderr = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.400851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.400951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_syslog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.401033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.401181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 00:57:42.401267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.401436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.401513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.401675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.401751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.401936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.402022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.402176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.402260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.402420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.402503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.402656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.403328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.403447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.403526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.403687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.403814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.403969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.404936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.405946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.406902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.407881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.408815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.409245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.409323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.409474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.409558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.409705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.409783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.409933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.410940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.411871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.412953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.413881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.414882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.415993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.416938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.417914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418635 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.418899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.419121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.419272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.419398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.419631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.419709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.419895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.420027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.420148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.420389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.420467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.420628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.420769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.420874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.421869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.422963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.423988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.424915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.425851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.external = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.426993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.password = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.token = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427815 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.external = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.427941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.password = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.token = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.428985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.429924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.430876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.431924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.432872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.433831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.434509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.434594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.434740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.434827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.434986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.435985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.436943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.437957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.438909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.439888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.440957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441756 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.441914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442000 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.442940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.443918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.444852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.445942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.446994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.447974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448699 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.448932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.449103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.449187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.449352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.449431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.449628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.450223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.450387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.450470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.450636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.450716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.450875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.450954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.451915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.452856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.453957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.454972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.455962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.456935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.457996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.458969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.459960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.460915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.461866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.462980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.463973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.464927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465862 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.465941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.466093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.466292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.466371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.466532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.466611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.466784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.466880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.467873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.468949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469747 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.469973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.470927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.471923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.472873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.473988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.474926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.475974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.476949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.477936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 00:57:42.478012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.478247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.478485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.478730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.478964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.479203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.479438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 00:57:42.479608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 00:57:42.507273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.507502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.507736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.508031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.508264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.508490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.516573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.517383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.544344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.544571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.545104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.545338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.600084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.601439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.648950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.649177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.649312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.649547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.804576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.804758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.804870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.804987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.805847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.828936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.829998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.830081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.830206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.830286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.830416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.841296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.841451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.853903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.854927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.855009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.855137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.855219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.855352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 00:57:42.873849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.874057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 00:57:42.874141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.874295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 00:57:42.874401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.874600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 00:57:42.874682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.874825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 00:57:42.874906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.875100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.875191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.875362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.875444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.875614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.875692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.875896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.885041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.885321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.885406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.885591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.885673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.885863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.885936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.886152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.892784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.893084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.895629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.895945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.896030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.896227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.896309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.896509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.896602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.896790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.896871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897664 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.897940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.898121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.898201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.898386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.898466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.898652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.921334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.921656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.922774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.923062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.923143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.923388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.927636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.927947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.928030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.928230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.928324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.928549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.934956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.935240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.935403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.935691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.940808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.941101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.941305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.941388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.941602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.941696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.941888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.941970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.942173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.942254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.942442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.942523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.942708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.942788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.942981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.943259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.957954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.958239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.958500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.958782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.962058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.962629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.965277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.965577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.965868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.966173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.969707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.970336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.972798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.973458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.975997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.976296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.976582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.976705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.976879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.977006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.977160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.977292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.977457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.977578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.977873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.978165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.993398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.993681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.993945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.994205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.994285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:42.994528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.994798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:42.995062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.001488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.002454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.003012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.004024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.004557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.004832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.005547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.005823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.007483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.007785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.008146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.008429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.008508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.008757 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.008838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.009044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.009329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.009725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.019757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.020055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.020335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.020613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.020829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.021002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.021117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.021284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.021403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.021569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.021686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.022066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.022348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.022630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.036891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.038098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.039740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.041007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.042563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.044380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.044535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.045825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.046003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.046282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.046677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.047354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.047637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.048069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.052931 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.054350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.054433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.055832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.057224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.057621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.057908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.058184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.058679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.059656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.059970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.060249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.066363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.066668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.066950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.067947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.068248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.068532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.073697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.073990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.074548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.074845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.075124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.075362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 00:57:43.075453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.075648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.075729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.075942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.076023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.076219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.076310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.076496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.076577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.076763 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.076845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.077025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.077106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.077291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.077413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.077562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.077849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.078121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.078397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.078660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.078935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 00:57:43.110993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557827b769c0 pid: 81610 (default app) Sep 06 00:57:43.112723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557827b769c0 pid: 81611 (default app) Sep 06 00:57:43.114237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f96d3-56c8-4171-b4eb-9c58c8c07802 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:43.114531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f96d3-56c8-4171-b4eb-9c58c8c07802 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:43.114805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f96d3-56c8-4171-b4eb-9c58c8c07802 None None] PATH_INFO: `/v3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:43.118925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Tue Sep 6 00:57:43 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:57:49.697364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd0732-cea0-4c37-8596-fc194d3ee4ee None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:49.697864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd0732-cea0-4c37-8596-fc194d3ee4ee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:49.697951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd0732-cea0-4c37-8596-fc194d3ee4ee None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:49.702739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:57:49.707920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:49.708428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:49.708526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:49.777633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: INFO dbcounter [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] Registered counter for database keystone Sep 06 00:57:49.796962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG oslo_db.sqlalchemy.engines [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81610) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 00:57:49.803019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writer thread running {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 00:57:49.978237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG passlib.handlers.bcrypt [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81610) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 00:57:49.978620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG passlib.handlers.bcrypt [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81610) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 00:57:50.036254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:57:50.037849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:57:50.161915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:57:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 00:57:50.601374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd0732-cea0-4c37-8596-fc194d3ee4ee None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:50.671844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: INFO dbcounter [None req-25fd0732-cea0-4c37-8596-fc194d3ee4ee None None] Registered counter for database keystone Sep 06 00:57:50.692658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG oslo_db.sqlalchemy.engines [None req-25fd0732-cea0-4c37-8596-fc194d3ee4ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81611) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 00:57:50.698824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writer thread running {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 00:57:50.815717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd0732-cea0-4c37-8596-fc194d3ee4ee None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:50.820061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d37b9-1b97-4d98-a1d5-8f28a1ce72b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:50.820736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d37b9-1b97-4d98-a1d5-8f28a1ce72b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:50.821015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d37b9-1b97-4d98-a1d5-8f28a1ce72b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:50.821286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d37b9-1b97-4d98-a1d5-8f28a1ce72b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:50.829454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d37b9-1b97-4d98-a1d5-8f28a1ce72b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:50.880652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING keystone.common.rbac_enforcer.enforcer [None req-297d37b9-1b97-4d98-a1d5-8f28a1ce72b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 00:57:50.880949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d37b9-1b97-4d98-a1d5-8f28a1ce72b8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:50.890054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Sep 6 00:57:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 00:57:50.895974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:50.911026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e213-439b-45e3-97d1-3260d5dc78fe None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:50.915441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72769b4-3e3d-42c1-a9ee-9af2a01218c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:50.916588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72769b4-3e3d-42c1-a9ee-9af2a01218c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:50.916877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72769b4-3e3d-42c1-a9ee-9af2a01218c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:50.917157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72769b4-3e3d-42c1-a9ee-9af2a01218c0 admin admin] PATH_INFO: `/v3/projects/921fe46fc085463aa0816bc6ba906e3c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:50.918722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72769b4-3e3d-42c1-a9ee-9af2a01218c0 admin admin] RBAC: Authorizing `identity:get_project(project_id=921fe46fc085463aa0816bc6ba906e3c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:50.970641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f72769b4-3e3d-42c1-a9ee-9af2a01218c0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 00:57:50.970994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72769b4-3e3d-42c1-a9ee-9af2a01218c0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:50.975696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Tue Sep 6 00:57:50 2022] GET /identity/v3/projects/921fe46fc085463aa0816bc6ba906e3c => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:57:52.854757 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcdf53-de40-4aea-a4be-ffc158f503f6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:52.855289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcdf53-de40-4aea-a4be-ffc158f503f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:52.855289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcdf53-de40-4aea-a4be-ffc158f503f6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:52.859306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:57:52.864685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e1f8e-098a-4eb1-bd93-a75142881f99 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:52.865098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e1f8e-098a-4eb1-bd93-a75142881f99 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:52.865251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e1f8e-098a-4eb1-bd93-a75142881f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:52.884985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eb6e1f8e-098a-4eb1-bd93-a75142881f99 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:57:52.886518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eb6e1f8e-098a-4eb1-bd93-a75142881f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:57:52.894820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:57:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 00:57:53.161901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcdf53-de40-4aea-a4be-ffc158f503f6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:53.173681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcdf53-de40-4aea-a4be-ffc158f503f6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:53.177836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc845f58-687f-4757-9647-98dd5e2b3ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:53.178424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc845f58-687f-4757-9647-98dd5e2b3ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:53.178703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc845f58-687f-4757-9647-98dd5e2b3ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:53.178974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc845f58-687f-4757-9647-98dd5e2b3ad4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:53.186925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc845f58-687f-4757-9647-98dd5e2b3ad4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:53.188959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc845f58-687f-4757-9647-98dd5e2b3ad4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:53.197461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Sep 6 00:57:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 00:57:53.203186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e1f8e-098a-4eb1-bd93-a75142881f99 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:53.212247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e1f8e-098a-4eb1-bd93-a75142881f99 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:53.216611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f111546-60f2-4a38-9e2a-20ddf24bba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:53.217236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111546-60f2-4a38-9e2a-20ddf24bba0f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:53.217520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111546-60f2-4a38-9e2a-20ddf24bba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:53.217793 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111546-60f2-4a38-9e2a-20ddf24bba0f admin admin] PATH_INFO: `/v3/users/1842912f886f4b17b51cd35bebba1a7e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:53.219107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f111546-60f2-4a38-9e2a-20ddf24bba0f admin admin] RBAC: Authorizing `identity:get_user(user_id=1842912f886f4b17b51cd35bebba1a7e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:53.220960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f111546-60f2-4a38-9e2a-20ddf24bba0f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:53.222467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Tue Sep 6 00:57:53 2022] GET /identity/v3/users/1842912f886f4b17b51cd35bebba1a7e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:57:55.178857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94dcb-6d7b-4ca5-a945-d84a4419aec1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:55.179369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94dcb-6d7b-4ca5-a945-d84a4419aec1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:55.179460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94dcb-6d7b-4ca5-a945-d84a4419aec1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:55.180764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:57:55.185950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c14212-0a3a-45e6-a9ef-2a1126055276 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:55.186332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c14212-0a3a-45e6-a9ef-2a1126055276 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:55.186618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c14212-0a3a-45e6-a9ef-2a1126055276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:55.205684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb5e75-b47f-49fd-b5ea-11403407bfdf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:55.205974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb5e75-b47f-49fd-b5ea-11403407bfdf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:55.206256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb5e75-b47f-49fd-b5ea-11403407bfdf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:55.207584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:57:55.213526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:55.213817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:55.214094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:55.219799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-75c14212-0a3a-45e6-a9ef-2a1126055276 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:57:55.222849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-75c14212-0a3a-45e6-a9ef-2a1126055276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:57:55.257557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81611) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 00:57:55.257947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81611) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 00:57:55.302758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:57:55.307394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:57:55.309287 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:57:55.407304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:57:55.572609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c14212-0a3a-45e6-a9ef-2a1126055276 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:55.599026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:55.677215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c14212-0a3a-45e6-a9ef-2a1126055276 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:55.677790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:55.711216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:55.712080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:55.712363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:55.712641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:55.724094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:57:55.732933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:57:55.733848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:57:55.734744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:57:55.739684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:55.776832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a74a31-6362-45a6-bbdb-1965bf5e80da None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:55.779786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:55.817286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:55.818150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:55.818431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:55.818704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:55.820387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:55.839048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eadafe-f932-48d5-a720-cf2914adfd0a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:55.875768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:55.876390 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:55.876671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:55.876946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:55.877617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:57:55.877928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:55.879326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:55.886074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:55.887593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:55.891307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:57:55.896824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:55.984588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 00:57:55.990184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:55.992814 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:55.993158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925b3e2-f804-4c6f-b477-b7445d0065b1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:56.027337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:56.027999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:56.028282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:56.028583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] PATH_INFO: `/v3/users/1842912f886f4b17b51cd35bebba1a7e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:56.029718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] RBAC: Authorizing `identity:get_user(user_id=1842912f886f4b17b51cd35bebba1a7e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:56.031167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:56.032750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:57:55 2022] GET /identity/v3/users/1842912f886f4b17b51cd35bebba1a7e => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:57:56.038377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:56.053670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:56.087945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360934-e85b-4d68-a749-e7600ee79f80 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:56.120383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:56.121169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:56.121457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:56.121737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:56.136358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0f642-14fa-41e6-82f8-b397ea5b7c51 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:56.140953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:56.142418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:57:56.160417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:57:56.161338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:57:56.162146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:57:56.162146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:57:56.162146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:57:56.162146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:57:56.167468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:56.171008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:56.171643 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:56.171970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:56.172242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:56.173352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:56.174802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:56.176509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:57:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:57:56.181938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:56.224637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:56.261350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:56.268659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5232fb-6041-4857-bb4c-8f2f2d6ee349 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:56.280356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf54e84-7896-4f31-b9f7-fd7baee5c19a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:56.301759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e8d0f-cfb0-4685-bd81-857c73c1fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:56.302366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8d0f-cfb0-4685-bd81-857c73c1fc49 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:56.302652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8d0f-cfb0-4685-bd81-857c73c1fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:56.302929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8d0f-cfb0-4685-bd81-857c73c1fc49 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:56.303574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e8d0f-cfb0-4685-bd81-857c73c1fc49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:57:56.303916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e8d0f-cfb0-4685-bd81-857c73c1fc49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:56.305334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e8d0f-cfb0-4685-bd81-857c73c1fc49 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:56.314477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ed9ec-fcf4-41f7-9a6c-0146e53123f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:56.315074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ed9ec-fcf4-41f7-9a6c-0146e53123f4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:56.315354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ed9ec-fcf4-41f7-9a6c-0146e53123f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:56.315625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ed9ec-fcf4-41f7-9a6c-0146e53123f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:56.316377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ed9ec-fcf4-41f7-9a6c-0146e53123f4 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.domain.id=default, user.id=1842912f886f4b17b51cd35bebba1a7e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:57:56.316680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ed9ec-fcf4-41f7-9a6c-0146e53123f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:56.318089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ed9ec-fcf4-41f7-9a6c-0146e53123f4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:56.322682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:57:56.324606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Sep 6 00:57:56 2022] GET /identity/v3/role_assignments?user.id=1842912f886f4b17b51cd35bebba1a7e&scope.domain.id=default&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:57:58.493939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0dd51-63b7-454d-bbc7-4dc47a38639e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:58.494825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0dd51-63b7-454d-bbc7-4dc47a38639e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:58.495248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0dd51-63b7-454d-bbc7-4dc47a38639e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:58.497109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:57:58.502688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77735d-ede6-436e-adfc-c703cf318f1d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:58.503540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77735d-ede6-436e-adfc-c703cf318f1d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:58.504000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77735d-ede6-436e-adfc-c703cf318f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:58.511018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7c573-1298-4149-b318-53a01d7c3979 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:58.511802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7c573-1298-4149-b318-53a01d7c3979 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:58.512364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7c573-1298-4149-b318-53a01d7c3979 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:58.515478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 12/23] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:57:58.539353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84335-8c7f-4a42-a7ec-23faee49bc4f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:58.539931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84335-8c7f-4a42-a7ec-23faee49bc4f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:58.540350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84335-8c7f-4a42-a7ec-23faee49bc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:58.541420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cf77735d-ede6-436e-adfc-c703cf318f1d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:57:58.542786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cf77735d-ede6-436e-adfc-c703cf318f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:57:58.578218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c9c84335-8c7f-4a42-a7ec-23faee49bc4f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:57:58.581322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c9c84335-8c7f-4a42-a7ec-23faee49bc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:57:58.711390 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 12/24] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:57:58.761794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:57:58.968120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94642e0c-3ada-4be6-9a94-251264d89646 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:58.968815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94642e0c-3ada-4be6-9a94-251264d89646 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:58.969294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94642e0c-3ada-4be6-9a94-251264d89646 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:58.971106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 13/26] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:57:58.976848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f923d0-a7a4-44af-824e-bf52a8079ad0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:58.977180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f923d0-a7a4-44af-824e-bf52a8079ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:58.977462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f923d0-a7a4-44af-824e-bf52a8079ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:59.001171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-10f923d0-a7a4-44af-824e-bf52a8079ad0 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:57:59.002387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-10f923d0-a7a4-44af-824e-bf52a8079ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:57:59.057853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94642e0c-3ada-4be6-9a94-251264d89646 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:59.068808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:59.115381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 14/27] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:57:59.188648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:59.223454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94642e0c-3ada-4be6-9a94-251264d89646 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:59.270349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:59.270996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:59.271280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:59.271558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:59.287595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:59.289113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:57:59.308698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:57:59.309534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:57:59.310356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:57:59.310356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:57:59.310356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:57:59.310510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f923d0-a7a4-44af-824e-bf52a8079ad0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:59.326800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 14/28] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:57:59.332983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:59.491170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90382ee-05c6-492f-90c6-02e8d33ddd5c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:59.533200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:59.539970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f923d0-a7a4-44af-824e-bf52a8079ad0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:59.555435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:59.556367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:59.556838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:59.557319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:59.558149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:57:59.560607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:59.562191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:59.611446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 15/29] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:57:59.612324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:59.623957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077692f7-5b18-49d5-84b7-3f4c51a5c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:59.624899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077692f7-5b18-49d5-84b7-3f4c51a5c236 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:59.625371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077692f7-5b18-49d5-84b7-3f4c51a5c236 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:59.625986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077692f7-5b18-49d5-84b7-3f4c51a5c236 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:59.639193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077692f7-5b18-49d5-84b7-3f4c51a5c236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-077692f7-5b18-49d5-84b7-3f4c51a5c236 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:57:59.646435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:57:59.647340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:57:59.648120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:57:59.650854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 15/30] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:57:59.652677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72e096-a53b-4072-aba8-893bd285c002 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:59.653158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72e096-a53b-4072-aba8-893bd285c002 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:59.653738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72e096-a53b-4072-aba8-893bd285c002 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:59.655542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 16/31] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:57:59.656804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72e096-a53b-4072-aba8-893bd285c002 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:59.704561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:57:59.723665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957653af-0aec-4e08-b19e-297f043e6415 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:59.758582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:59.759160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:59.759450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:59.759726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:59.760429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:59.761821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:59.782378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72e096-a53b-4072-aba8-893bd285c002 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:57:59.820497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4d28-9d12-448f-86fb-313906669dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:57:59.821524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4d28-9d12-448f-86fb-313906669dcd None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:57:59.822026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4d28-9d12-448f-86fb-313906669dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:57:59.822488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4d28-9d12-448f-86fb-313906669dcd None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:57:59.823538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4d28-9d12-448f-86fb-313906669dcd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:57:59.825271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4d28-9d12-448f-86fb-313906669dcd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:57:59.907773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 16/32] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 6 00:57:59 2022] POST /identity/v3/roles => generated 225 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:57:59.907773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:57:59.960827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:00.015703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 17/33] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:57:59 2022] POST /identity/v3/projects => generated 301 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:00.025873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cb686-d2d6-4feb-bbf2-94d62b2ff398 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.026155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e612c-6835-4905-b807-28b766fb2a69 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:00.026604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cb686-d2d6-4feb-bbf2-94d62b2ff398 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.027033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cb686-d2d6-4feb-bbf2-94d62b2ff398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.061691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e43cb686-d2d6-4feb-bbf2-94d62b2ff398 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:00.063663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e43cb686-d2d6-4feb-bbf2-94d62b2ff398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:00.069404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78511c-8c27-4e43-92a6-dba33c8902e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:00.070203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78511c-8c27-4e43-92a6-dba33c8902e0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.070684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78511c-8c27-4e43-92a6-dba33c8902e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.071040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78511c-8c27-4e43-92a6-dba33c8902e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.074666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78511c-8c27-4e43-92a6-dba33c8902e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:00.074666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78511c-8c27-4e43-92a6-dba33c8902e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:00.074666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78511c-8c27-4e43-92a6-dba33c8902e0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:00.089941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 17/34] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:00.089941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c95c4-96f1-4e70-aafd-5dc30a675a88 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.089941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c95c4-96f1-4e70-aafd-5dc30a675a88 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.089941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c95c4-96f1-4e70-aafd-5dc30a675a88 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.089941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 18/35] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:00.095728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48dec9-076a-4200-9b63-ff272acb100a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.096159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48dec9-076a-4200-9b63-ff272acb100a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.096753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48dec9-076a-4200-9b63-ff272acb100a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.118604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8c48dec9-076a-4200-9b63-ff272acb100a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:00.120539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8c48dec9-076a-4200-9b63-ff272acb100a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:00.155369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 18/36] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:00.156561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cb686-d2d6-4feb-bbf2-94d62b2ff398 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:00.205881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:00.235941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 19/37] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:00.237386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728c683-f128-4030-9376-99eb84ff4cfd None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.237676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728c683-f128-4030-9376-99eb84ff4cfd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.237955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728c683-f128-4030-9376-99eb84ff4cfd None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.239208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 20/38] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:00.245205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031433c-011e-48f2-a9a8-e51974a6a0de None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.245496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031433c-011e-48f2-a9a8-e51974a6a0de None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.245776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031433c-011e-48f2-a9a8-e51974a6a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.269274 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8031433c-011e-48f2-a9a8-e51974a6a0de None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:00.270904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8031433c-011e-48f2-a9a8-e51974a6a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:00.300576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cb686-d2d6-4feb-bbf2-94d62b2ff398 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:00.311941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:00.343615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:00.344647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.345156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.345621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] PATH_INFO: `/v3/users/1842912f886f4b17b51cd35bebba1a7e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.348206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] RBAC: Authorizing `identity:get_user(user_id=1842912f886f4b17b51cd35bebba1a7e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:00.349860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:00.351686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 19/39] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:00 2022] GET /identity/v3/users/1842912f886f4b17b51cd35bebba1a7e => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:00.358233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:00.471178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:00.490995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b64ab-d091-416e-ab95-22b092339b5b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:00.506462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 21/40] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:00.507455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031433c-011e-48f2-a9a8-e51974a6a0de None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:00.525387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:00.526015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.526453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.526734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.528012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:00.529981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:00.531637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 20/41] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:00.538014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:00.626080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031433c-011e-48f2-a9a8-e51974a6a0de None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:00.661161 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a471f-bc90-423d-8232-c734f7e3c327 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:00.662779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:00.663398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.663691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.663998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.664670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:00.666052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:00.680825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 22/42] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 6 00:58:00 2022] POST /identity/v3/roles => generated 219 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:00.681627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:00.707241 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:00.708049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.708453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.708743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] PATH_INFO: `/v3/domains/default/users/1842912f886f4b17b51cd35bebba1a7e/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.711152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1842912f886f4b17b51cd35bebba1a7e, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:00.712851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:00.728591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:00.742305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 21/43] 10.222.0.23 () {58 vars in 1361 bytes} [Tue Sep 6 00:58:00 2022] PUT /identity/v3/domains/default/users/1842912f886f4b17b51cd35bebba1a7e/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 00:58:00.798505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:00.818158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:00.839007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4082433-2c18-4f64-b574-43cb67703ee4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:00.879373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:00.879952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.880241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.880520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.881293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:00.882737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:00.902221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 23/44] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 6 00:58:00 2022] POST /identity/v3/roles => generated 223 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:00.934874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac25625-044b-458d-a786-ed08a2ad0c56 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:00.975319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c46f3d-9a0a-42fc-871c-3efc50eaedd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:00.976049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c46f3d-9a0a-42fc-871c-3efc50eaedd6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:00.976336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c46f3d-9a0a-42fc-871c-3efc50eaedd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:00.976639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c46f3d-9a0a-42fc-871c-3efc50eaedd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:00.977825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c46f3d-9a0a-42fc-871c-3efc50eaedd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:00.979318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c46f3d-9a0a-42fc-871c-3efc50eaedd6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:00.981090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 22/45] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:00.987172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:01.082526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:01.096724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdec21-69da-4ea2-8138-523a65f68e39 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:01.132752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413dd8c-0e6e-4cfa-abe2-8bdbfbc56b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:01.133300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413dd8c-0e6e-4cfa-abe2-8bdbfbc56b12 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:01.133583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413dd8c-0e6e-4cfa-abe2-8bdbfbc56b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:01.133861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413dd8c-0e6e-4cfa-abe2-8bdbfbc56b12 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:01.134512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413dd8c-0e6e-4cfa-abe2-8bdbfbc56b12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:01.135998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413dd8c-0e6e-4cfa-abe2-8bdbfbc56b12 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:01.166407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 24/46] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:58:00 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:02.744673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a8e88-ac27-4a8b-9dc9-c88e44dcc370 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:02.745194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a8e88-ac27-4a8b-9dc9-c88e44dcc370 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:02.745194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a8e88-ac27-4a8b-9dc9-c88e44dcc370 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:02.746537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 23/47] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:02.752229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df8ee-548d-4770-a2b2-8389b4fd3028 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:02.752638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df8ee-548d-4770-a2b2-8389b4fd3028 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:02.752785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df8ee-548d-4770-a2b2-8389b4fd3028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:02.771002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-365df8ee-548d-4770-a2b2-8389b4fd3028 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:02.772191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-365df8ee-548d-4770-a2b2-8389b4fd3028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:02.836512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 25/48] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:03.053996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9450b-02e0-4b56-a487-24b9d6f312df None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.054294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9450b-02e0-4b56-a487-24b9d6f312df None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.054575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9450b-02e0-4b56-a487-24b9d6f312df None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.056011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 24/49] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:03.061707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0cf85-3daa-421f-915d-35fae4698b78 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.062004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0cf85-3daa-421f-915d-35fae4698b78 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.062287 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0cf85-3daa-421f-915d-35fae4698b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.080930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5ce0cf85-3daa-421f-915d-35fae4698b78 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:03.082111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5ce0cf85-3daa-421f-915d-35fae4698b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:03.120346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9450b-02e0-4b56-a487-24b9d6f312df None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:03.143642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:03.151777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 26/50] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:03.205028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:03.228664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9450b-02e0-4b56-a487-24b9d6f312df None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:03.264147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:03.264775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.265053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.265326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.275075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:03.280023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:03.280829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:03.281596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 25/51] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:58:03.287170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0cf85-3daa-421f-915d-35fae4698b78 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:03.384549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0cf85-3daa-421f-915d-35fae4698b78 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:03.417690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:03.418295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.418573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.418843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.420116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:03.420418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:03.421782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:03.429559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 27/52] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:03.431535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:03.434760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:03.493984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:03.533282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58350200-870a-41f2-b652-e558ee2f4863 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:03.537524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2753ea-6f3e-45ce-b2a2-a3d21ecbd979 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:03.568194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:03.568853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.569139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.569419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] PATH_INFO: `/v3/users/1842912f886f4b17b51cd35bebba1a7e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.570639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] RBAC: Authorizing `identity:get_user(user_id=1842912f886f4b17b51cd35bebba1a7e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:03.572189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:03.573846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 28/53] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:03 2022] GET /identity/v3/users/1842912f886f4b17b51cd35bebba1a7e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:03.579655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:03.674513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:03.675142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.675427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.675701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.676910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:03.678367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:03.679975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 26/54] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:03.682885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c67d3-82dc-4578-8978-897f441020b9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:03.685397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:03.697610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:03.712109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:03.718745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:03.719378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.719662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.719971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.721087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:03.722550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:03.724143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 29/55] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:03.730142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:03.786358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce73377-4097-48d3-8e2e-62cbe4c7e6e0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:03.819251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4a79a-02a3-4fad-8fe7-e2526f9c9a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:03.819888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4a79a-02a3-4fad-8fe7-e2526f9c9a6e None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.820170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4a79a-02a3-4fad-8fe7-e2526f9c9a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.820452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4a79a-02a3-4fad-8fe7-e2526f9c9a6e None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.821090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4a79a-02a3-4fad-8fe7-e2526f9c9a6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:03.822518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4a79a-02a3-4fad-8fe7-e2526f9c9a6e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:03.829090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791d2c1-9670-4183-9b70-fbf8d097947b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:03.849163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 27/56] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Sep 6 00:58:03 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:58:03.870765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767fb0-05ae-49d4-9bcb-1f6732a78dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:03.871400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767fb0-05ae-49d4-9bcb-1f6732a78dec None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:03.871685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767fb0-05ae-49d4-9bcb-1f6732a78dec None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:03.871991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767fb0-05ae-49d4-9bcb-1f6732a78dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:03.872791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec767fb0-05ae-49d4-9bcb-1f6732a78dec None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.domain.id=default, user.id=1842912f886f4b17b51cd35bebba1a7e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:03.873105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec767fb0-05ae-49d4-9bcb-1f6732a78dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:03.874599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec767fb0-05ae-49d4-9bcb-1f6732a78dec None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:03.881447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 30/57] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Sep 6 00:58:03 2022] GET /identity/v3/role_assignments?user.id=1842912f886f4b17b51cd35bebba1a7e&scope.domain.id=default&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:05.757330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366517-b5fd-4f10-acf6-c10ffb0c4c08 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:05.757847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366517-b5fd-4f10-acf6-c10ffb0c4c08 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:05.757935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366517-b5fd-4f10-acf6-c10ffb0c4c08 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:05.759193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 28/58] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:05.764921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c3399-0fa1-4d11-9326-2c7e74b0c033 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:05.765280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c3399-0fa1-4d11-9326-2c7e74b0c033 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:05.765474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c3399-0fa1-4d11-9326-2c7e74b0c033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:05.783383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0c6c3399-0fa1-4d11-9326-2c7e74b0c033 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:05.784605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0c6c3399-0fa1-4d11-9326-2c7e74b0c033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:05.839320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:05.849681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 31/59] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:06.123561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366517-b5fd-4f10-acf6-c10ffb0c4c08 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:06.183813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:06.222888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366517-b5fd-4f10-acf6-c10ffb0c4c08 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:06.255915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cd23d-763b-4937-bfd0-281592f6067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:06.256535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cd23d-763b-4937-bfd0-281592f6067d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:06.256816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cd23d-763b-4937-bfd0-281592f6067d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:06.257088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cd23d-763b-4937-bfd0-281592f6067d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:06.258191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cd23d-763b-4937-bfd0-281592f6067d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:06.259627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cd23d-763b-4937-bfd0-281592f6067d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:06.261211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 29/60] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:06.266678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c3399-0fa1-4d11-9326-2c7e74b0c033 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:06.364270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c3399-0fa1-4d11-9326-2c7e74b0c033 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:06.396472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cde550-c53b-4048-a7d4-b66769374f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:06.397020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cde550-c53b-4048-a7d4-b66769374f20 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:06.397304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cde550-c53b-4048-a7d4-b66769374f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:06.397577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cde550-c53b-4048-a7d4-b66769374f20 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:06.398221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cde550-c53b-4048-a7d4-b66769374f20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:06.399651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cde550-c53b-4048-a7d4-b66769374f20 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:06.425633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 32/61] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:58:06 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:09.502260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110d922-4189-42bb-844e-7144ffbeb933 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.503280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110d922-4189-42bb-844e-7144ffbeb933 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.503784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110d922-4189-42bb-844e-7144ffbeb933 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.505337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 30/62] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:09.510459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec904-735e-4d92-847e-e69455887601 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.511269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec904-735e-4d92-847e-e69455887601 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.511768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec904-735e-4d92-847e-e69455887601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.540762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-20dec904-735e-4d92-847e-e69455887601 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:09.541973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-20dec904-735e-4d92-847e-e69455887601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:09.611313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:09.621946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7ec7-1607-4498-9b1c-c46e715ae3d5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.622544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7ec7-1607-4498-9b1c-c46e715ae3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.623325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7ec7-1607-4498-9b1c-c46e715ae3d5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.625466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 31/63] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:09.630792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b46cd1-98c7-43e8-b92d-e166082fdd35 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.632027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b46cd1-98c7-43e8-b92d-e166082fdd35 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.632541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b46cd1-98c7-43e8-b92d-e166082fdd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.648883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 33/64] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:09.653328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-39b46cd1-98c7-43e8-b92d-e166082fdd35 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:09.655326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-39b46cd1-98c7-43e8-b92d-e166082fdd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:09.758285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 32/65] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:09.818768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf19da-7345-40f8-88c1-dc64090b9b82 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.819077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf19da-7345-40f8-88c1-dc64090b9b82 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.819334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf19da-7345-40f8-88c1-dc64090b9b82 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.820722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 34/66] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:09.826474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120ffe-b230-4db4-9691-deb4430c72e5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.826773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120ffe-b230-4db4-9691-deb4430c72e5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.827056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120ffe-b230-4db4-9691-deb4430c72e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.851382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8a120ffe-b230-4db4-9691-deb4430c72e5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:09.852066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a6d8-6583-4b34-9bdc-9443868db2d7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.852360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a6d8-6583-4b34-9bdc-9443868db2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.852640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a6d8-6583-4b34-9bdc-9443868db2d7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.852733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a120ffe-b230-4db4-9691-deb4430c72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:09.853993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 35/67] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:09.860671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272331dd-85e8-446b-bcd4-a0abcfbe3643 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:09.860965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272331dd-85e8-446b-bcd4-a0abcfbe3643 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:09.861239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272331dd-85e8-446b-bcd4-a0abcfbe3643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:09.889769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-272331dd-85e8-446b-bcd4-a0abcfbe3643 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:09.890959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-272331dd-85e8-446b-bcd4-a0abcfbe3643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:09.947346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:09.970933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 33/68] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:09.996796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a120ffe-b230-4db4-9691-deb4430c72e5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:09.997976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 36/69] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:10.104084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c532-2024-43d9-ac2c-2957f234ee32 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.104725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c532-2024-43d9-ac2c-2957f234ee32 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.105526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c532-2024-43d9-ac2c-2957f234ee32 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.107244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 37/70] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:10.112262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab204e-f6b1-41f7-88da-6cef30ce120e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.112828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab204e-f6b1-41f7-88da-6cef30ce120e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.113282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab204e-f6b1-41f7-88da-6cef30ce120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.147863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-46ab204e-f6b1-41f7-88da-6cef30ce120e None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:10.149063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-46ab204e-f6b1-41f7-88da-6cef30ce120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:10.166480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a120ffe-b230-4db4-9691-deb4430c72e5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.204140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.204782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.205070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.205335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.216606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:10.221996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:10.222867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:10.223606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:10.230820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 34/71] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:10.231847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.232785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 38/72] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:10.234057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab204e-f6b1-41f7-88da-6cef30ce120e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.280183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:10.292052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:10.341761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab204e-f6b1-41f7-88da-6cef30ce120e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.355052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f21920-1091-46b1-ad94-958f926ff61f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.376161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.376801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.377502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.377788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.378468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:10.378757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:10.380181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:10.389384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 39/73] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:10.390399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.396634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.397282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.397568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.397843 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.408520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:10.413606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:10.414424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:10.415151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 35/74] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:10.415841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.502020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:10.505818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ec9a8-1a45-4b33-a429-e9e1e2c2c5fb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.519662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08dc44-ae44-44f5-947d-05bb4ef98753 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.530602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:10.541157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.541795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.542080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.542359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.552977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:10.558359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:10.559224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:10.560008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 40/75] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:10.561242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.580147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.580791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.581113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.581997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.592229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:10.597101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:10.598132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:10.599086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:10.603044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 36/76] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:10.604691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.668705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904d400-66da-4b7a-9bc2-33cffa35ac63 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.702052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.702682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.702966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.703245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.703503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b538c43-c4f9-4d99-8879-f254639edb34 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.716323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:10.717830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:10.719436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 41/77] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:10.720231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.738640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:10.738919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.739533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.739892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.740176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.740840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:10.741143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:10.742521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:10.754534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 37/78] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:10.755319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.785204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:10.822815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3d1f-cc70-484b-a953-b3f68cfd3e5c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.851527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee596b-5851-48a9-bdff-f16ab7069076 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:10.854963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.855597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.855912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.856193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.857813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:10.859351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:10.860981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 42/79] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:10.862936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.895981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:10.896595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:10.896881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:10.897157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:10.897823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:10.898133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:10.899511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:10.907885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 38/80] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:10.908679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:10.955906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:10.972188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1aea47-9d46-4877-9406-15d7da5f5983 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.000339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.005567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.006177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.006456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.006733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.007399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:11.007489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f474775-e5a5-4292-881b-15d49ced528e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.007701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.009117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.017141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 43/81] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:11.021089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.041653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.042271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.042560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.042836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.109819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.111320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.118051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e21630-413a-4ba6-9bec-0262993c6d20 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.151309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.151955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.152244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.152528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] PATH_INFO: `/v3/users/1842912f886f4b17b51cd35bebba1a7e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.153644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] RBAC: Authorizing `identity:get_user(user_id=1842912f886f4b17b51cd35bebba1a7e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.155601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.157216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 44/82] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:11 2022] GET /identity/v3/users/1842912f886f4b17b51cd35bebba1a7e => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:11.157956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.176727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.206160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.213730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 39/83] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:11.214707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.260621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056362-d250-4d73-9f8c-956b3e3c3705 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.293741 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.294298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.294614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.294892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.295542 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.297029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.312667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cc0b3-02db-40b0-9cba-b4357a18f0fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.318014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 45/84] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:58:11 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:11.320710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.362397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.363051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.363340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.363617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.364989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.366535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.373468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 40/85] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:11 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:11.373468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.409011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.445038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c652b-cb09-42cb-8689-7aea0d5152fc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.448173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.491206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.492231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.492823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.493115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.494430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.495577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce23c5-809b-42ee-912d-f69432cfbdd6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.495949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.497772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 46/86] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:11 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:11.499556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.544341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.545006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.545285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.547142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.624802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b28572-a0f6-44da-ae48-c265d58fc015 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.644864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.646359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.666591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.667238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.667522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.667825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.669076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.670573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.672494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 47/87] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:11 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:11.673266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.685008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.709475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.750852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 41/88] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:11.751784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:11.782482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b15348-7466-4f9f-be52-b564c01312e9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.819082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.819790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.820118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.820356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.854036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcb0b5-78ff-4e03-b910-aadfca34b058 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:11.890682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:11.891401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:11.891777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:11.892156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:11.893064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.894723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:11.908139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.923068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:11.966265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:11.968329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.018151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 48/89] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:12.019412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:12.078362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 42/90] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:12.079584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:12.141052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ba461-540e-49e2-9e7b-dd3f9d86dd13 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:12.147570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:12.152594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:12.183447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:12.184459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:12.184815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:12.185898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:12.187020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=47aae071f79647d0a619a13ffb7ae739) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:12.187541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:12.189361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.192829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b90f-9254-4285-a449-5315a08caf1a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:12.196572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 49/91] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:12 2022] GET /identity/v3/role_assignments?user.id=47aae071f79647d0a619a13ffb7ae739&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:12.197298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:12.271958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:12.272538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:12.272823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:12.273102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:12.276346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:12.277863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.279863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 43/92] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:12 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:12.280470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:12.393513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648aa44e-0516-4f87-a16e-4f9eb02cc1f5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:12.414416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f1dea-57ff-49b3-9709-d06de2db3883 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:12.436572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:12.437215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:12.437499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:12.437775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:12.439016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:12.440579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.442427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 50/93] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:12 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:12.447021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:12.447803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:12.458484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:12.459132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:12.459420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:12.459694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] PATH_INFO: `/v3/projects/624580f535f14a4e883a2cedc84e9663` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:12.460890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] RBAC: Authorizing `identity:get_project(project_id=624580f535f14a4e883a2cedc84e9663)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:12.462483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.464589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 44/94] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:12 2022] GET /identity/v3/projects/624580f535f14a4e883a2cedc84e9663 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:12.465624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:12.499515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:12.616818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336230da-b548-469f-8d3a-3a9a6bfda1a1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:12.635368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aac15-3f12-454c-a093-f496ad9eff9f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:12.683198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:12.683873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:12.684132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:12.684407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:12.685149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=1842912f886f4b17b51cd35bebba1a7e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:12.685514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:12.687775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.698741 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 51/95] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:12 2022] GET /identity/v3/role_assignments?user.id=1842912f886f4b17b51cd35bebba1a7e&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:12.700231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab8132-235d-4448-b9e3-d63de4a54c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:12.700890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab8132-235d-4448-b9e3-d63de4a54c04 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:12.701176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab8132-235d-4448-b9e3-d63de4a54c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:12.701450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab8132-235d-4448-b9e3-d63de4a54c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:12.702362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:12.703854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab8132-235d-4448-b9e3-d63de4a54c04 None admin] RBAC: Adding query filter params (role.id=66704401130047e78e714312f45defd9, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=47aae071f79647d0a619a13ffb7ae739) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:12.704174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab8132-235d-4448-b9e3-d63de4a54c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:12.707856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab8132-235d-4448-b9e3-d63de4a54c04 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.729145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 45/96] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:12 2022] GET /identity/v3/role_assignments?user.id=47aae071f79647d0a619a13ffb7ae739&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=66704401130047e78e714312f45defd9 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:12.865683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5385a0-37fc-4bd5-abee-988f09daa8aa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:12.873097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:12.930137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a6f6a-ff85-44b0-9ecd-0e2e076f8e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:12.930769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6f6a-ff85-44b0-9ecd-0e2e076f8e71 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:12.931115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6f6a-ff85-44b0-9ecd-0e2e076f8e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:12.931399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a6f6a-ff85-44b0-9ecd-0e2e076f8e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:12.932172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a6f6a-ff85-44b0-9ecd-0e2e076f8e71 None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=624580f535f14a4e883a2cedc84e9663, user.id=47aae071f79647d0a619a13ffb7ae739) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:12.932556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a6f6a-ff85-44b0-9ecd-0e2e076f8e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:12.934059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a6f6a-ff85-44b0-9ecd-0e2e076f8e71 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:12.940604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 52/97] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:12 2022] GET /identity/v3/role_assignments?user.id=47aae071f79647d0a619a13ffb7ae739&scope.project.id=624580f535f14a4e883a2cedc84e9663&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:13.567773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bd207-6966-4523-b397-0f1b68194aa4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:13.568319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bd207-6966-4523-b397-0f1b68194aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:13.568859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bd207-6966-4523-b397-0f1b68194aa4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:13.570228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 46/98] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:13.588068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e2b51-39c8-40db-ba00-6d6dd2b2d012 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:13.588361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e2b51-39c8-40db-ba00-6d6dd2b2d012 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:13.588640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e2b51-39c8-40db-ba00-6d6dd2b2d012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:13.615265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-979e2b51-39c8-40db-ba00-6d6dd2b2d012 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:13.616506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-979e2b51-39c8-40db-ba00-6d6dd2b2d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:13.703825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 53/99] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:13.728561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047dace-7215-441f-98ef-d84a446e9399 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:13.729037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047dace-7215-441f-98ef-d84a446e9399 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:13.729248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047dace-7215-441f-98ef-d84a446e9399 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:13.730552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 47/100] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:13.741673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b9607-2f0e-4305-85a4-0bec955a44b9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:13.742156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b9607-2f0e-4305-85a4-0bec955a44b9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:13.742574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b9607-2f0e-4305-85a4-0bec955a44b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:13.767744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0e4b9607-2f0e-4305-85a4-0bec955a44b9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:13.769210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e4b9607-2f0e-4305-85a4-0bec955a44b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:13.863769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 54/101] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:14.111144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047dace-7215-441f-98ef-d84a446e9399 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:14.180919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:14.237462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047dace-7215-441f-98ef-d84a446e9399 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:14.277946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:14.279008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:14.279508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:14.280053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:14.296607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:14.301964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:14.302873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:14.303662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:14.306239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 48/102] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:14.308907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b9607-2f0e-4305-85a4-0bec955a44b9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:14.316890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:14.369026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:14.464646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b9607-2f0e-4305-85a4-0bec955a44b9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:14.492658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec0785-d350-4b2e-8f6e-16a4abd3975a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:14.511616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:14.512303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:14.512588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:14.512864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:14.516410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:14.517878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:14.519483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 55/103] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:14.533106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:14.534466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:14.570047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:14.570672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:14.570957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:14.571238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:14.571950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:14.572261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:14.575100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:14.596992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 49/104] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:14.602516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:14.760227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:14.769859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a81fd-bfda-4187-837c-2ac1abf34897 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:14.824842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a819cbc-6d5f-4ca4-89b4-a843896725f1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:14.842185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:14.843082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:14.843666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:14.845432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:14.846575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:14.848262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:14.905825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 56/105] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:58:14 2022] POST /identity/v3/projects => generated 302 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:58:14.912824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdab5d-d30e-43ee-95cf-b2d3bb2d36ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:14.913812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdab5d-d30e-43ee-95cf-b2d3bb2d36ad None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:14.914344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdab5d-d30e-43ee-95cf-b2d3bb2d36ad None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:14.914812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdab5d-d30e-43ee-95cf-b2d3bb2d36ad None admin] PATH_INFO: `/v3/users/b7122418202c4c39a6f4095fbcdb9ca0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:14.939071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdab5d-d30e-43ee-95cf-b2d3bb2d36ad None admin] RBAC: Authorizing `identity:get_user(user_id=b7122418202c4c39a6f4095fbcdb9ca0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:14.940955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdab5d-d30e-43ee-95cf-b2d3bb2d36ad None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:14.942993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 50/106] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:14 2022] GET /identity/v3/users/b7122418202c4c39a6f4095fbcdb9ca0 => generated 289 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:14.948751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:15.077752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95466a3b-826f-48b6-b65e-d8a7cfd44cbd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:15.119366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:15.120404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.120956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.121436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.151254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:15.213301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c008f-380e-479e-9af5-19bbcfa1d8bf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.213629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c008f-380e-479e-9af5-19bbcfa1d8bf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.213883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c008f-380e-479e-9af5-19bbcfa1d8bf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.215213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 51/107] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:15.221570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:15.223340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:15.233548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b6c2-2c2f-4836-8064-50fc201177e1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.233548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b6c2-2c2f-4836-8064-50fc201177e1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.233886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b6c2-2c2f-4836-8064-50fc201177e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.272783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f355b6c2-2c2f-4836-8064-50fc201177e1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:15.274007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f355b6c2-2c2f-4836-8064-50fc201177e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:15.330444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 57/108] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:15.335937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:15.340818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:15.375829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 52/109] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:15.405891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:15.466095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3a56c-155d-4fc3-9142-269e66590023 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:15.509041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357e15e-c27f-4099-b1ee-18db559cd129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:15.509698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357e15e-c27f-4099-b1ee-18db559cd129 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.509985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357e15e-c27f-4099-b1ee-18db559cd129 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.510259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357e15e-c27f-4099-b1ee-18db559cd129 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.511421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357e15e-c27f-4099-b1ee-18db559cd129 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:15.512919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357e15e-c27f-4099-b1ee-18db559cd129 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:15.515555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 58/110] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:15 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:15.524854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355b6c2-2c2f-4836-8064-50fc201177e1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:15.629272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67914e-5ae4-4ead-8bb9-d02d73b7ed6c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.629588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67914e-5ae4-4ead-8bb9-d02d73b7ed6c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.629871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67914e-5ae4-4ead-8bb9-d02d73b7ed6c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.631599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 59/111] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:15.641389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7ce3-202b-4f3f-9794-8ca1d389387b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.641673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7ce3-202b-4f3f-9794-8ca1d389387b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.641952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7ce3-202b-4f3f-9794-8ca1d389387b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.664062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2ffc7ce3-202b-4f3f-9794-8ca1d389387b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:15.669521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2ffc7ce3-202b-4f3f-9794-8ca1d389387b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:15.713933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355b6c2-2c2f-4836-8064-50fc201177e1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:15.766317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 60/112] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:15.784195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7ce3-202b-4f3f-9794-8ca1d389387b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:15.804480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e7aa-f10b-457d-8e36-6e6fe4d964ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:15.805112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e7aa-f10b-457d-8e36-6e6fe4d964ff None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.805405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e7aa-f10b-457d-8e36-6e6fe4d964ff None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.805686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e7aa-f10b-457d-8e36-6e6fe4d964ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.806427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e7aa-f10b-457d-8e36-6e6fe4d964ff None admin] RBAC: Adding query filter params (role.id=b6eb85a18e7a483d859c5d57f92888b8, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=b7122418202c4c39a6f4095fbcdb9ca0) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:15.806751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e7aa-f10b-457d-8e36-6e6fe4d964ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:15.808225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e7aa-f10b-457d-8e36-6e6fe4d964ff None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:15.821645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 53/113] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:15 2022] GET /identity/v3/role_assignments?user.id=b7122418202c4c39a6f4095fbcdb9ca0&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=b6eb85a18e7a483d859c5d57f92888b8 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:15.848262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:15.919277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7ce3-202b-4f3f-9794-8ca1d389387b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:15.924298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029c264-d51f-4909-9003-45edf8cb28b2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.924788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029c264-d51f-4909-9003-45edf8cb28b2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.924859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029c264-d51f-4909-9003-45edf8cb28b2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.926212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 54/114] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:15.931286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04137fbe-7b7e-482d-be9d-700b28a6ecca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.931579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04137fbe-7b7e-482d-be9d-700b28a6ecca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.931893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04137fbe-7b7e-482d-be9d-700b28a6ecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.951463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-04137fbe-7b7e-482d-be9d-700b28a6ecca None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:15.952715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04137fbe-7b7e-482d-be9d-700b28a6ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:15.988597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077452b-f1bb-4b81-9ae8-15971269b3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:15.989272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077452b-f1bb-4b81-9ae8-15971269b3cb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:15.989560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077452b-f1bb-4b81-9ae8-15971269b3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:15.989842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077452b-f1bb-4b81-9ae8-15971269b3cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:15.990899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:16.016959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077452b-f1bb-4b81-9ae8-15971269b3cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0077452b-f1bb-4b81-9ae8-15971269b3cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:16.021993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:16.022879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:16.024087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 61/115] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:16.024986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d0cda-3967-4208-8eb0-4b28e9ef9f36 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.024986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d0cda-3967-4208-8eb0-4b28e9ef9f36 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.024986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d0cda-3967-4208-8eb0-4b28e9ef9f36 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.026705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 62/116] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:16.038675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d0cda-3967-4208-8eb0-4b28e9ef9f36 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.053168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 55/117] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:16.055970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0dcfd-5e58-4166-9c26-bf79f058b028 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.055970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0dcfd-5e58-4166-9c26-bf79f058b028 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.055970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0dcfd-5e58-4166-9c26-bf79f058b028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.092861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f1c0dcfd-5e58-4166-9c26-bf79f058b028 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:16.094076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f1c0dcfd-5e58-4166-9c26-bf79f058b028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:16.171870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 56/118] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:16.172551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0dcfd-5e58-4166-9c26-bf79f058b028 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.254737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d0cda-3967-4208-8eb0-4b28e9ef9f36 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.262682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:16.275946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:16.284880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0dcfd-5e58-4166-9c26-bf79f058b028 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.304708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.305323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.305605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.305933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.306605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:16.306906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:16.308406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:16.330272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 63/119] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:16.333856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.363639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.364315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.364644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.364921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.382573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:16.388196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:16.389078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:16.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 57/120] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:16.396781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.531414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0b2d2-3963-4926-992d-ac7e24f7a8e4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.546941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6995103-8d0d-44a6-aea5-be1254a1c182 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.579507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.580155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.580435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.580705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.581375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:16.581673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:16.583102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:16.593295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 58/121] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:16.594087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.598142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.599053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.599557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.600055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.601423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:16.603084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:16.604902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 64/122] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:16.605949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.618541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:16.642342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:16.710285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f759f5-d56c-4c25-8d50-91cc6d5c41fc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.714024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ee7e9-bbcc-4d04-a9d9-95ac77bb00af None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.746009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.746798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.747189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.747593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.752009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.753127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.753573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.754023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.758926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:16.764655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:16.765544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:16.766281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 65/123] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:16.767322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.769432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:16.775381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:16.776296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:16.777063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:16.779029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 59/124] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:16.780593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.865786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:16.869798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:16.876264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0565ca6-55c8-4582-8cfd-78ae31efd1ee None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.896184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f62b2b-0327-4141-abb5-60a42d577985 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:16.914369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.915090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.915445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.915820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] PATH_INFO: `/v3/users/1842912f886f4b17b51cd35bebba1a7e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.917043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] RBAC: Authorizing `identity:get_user(user_id=1842912f886f4b17b51cd35bebba1a7e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:16.918579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:16.920293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 66/125] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/users/1842912f886f4b17b51cd35bebba1a7e => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:16.921145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:16.934727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:16.936601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:16.937128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:16.937610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:16.938928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:16.940788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:16.942660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 60/126] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:16.948169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.023687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825a412-5863-41fa-835e-7beffd3a9ef6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.058140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.058845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.059210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.059566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.060252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efccd8-7b30-40e9-a2f8-f8be1e84865d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.060871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:17.061226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.062737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.071197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 67/127] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:17.072130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.100030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:17.101673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.102450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.102881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.103299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.104222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:17.104773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.106585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.116083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 61/128] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:17.117228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.129736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:17.180579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6395b7-faed-4d9a-bd21-6c7e14caf9ff None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.229354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81b7c1-4234-41ab-8d84-fa94088fabd4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.229766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.230407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.230689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.230960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.232421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.234791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.236527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 68/129] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:17 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:17.241318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.277852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.278520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.278802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.279109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/47aae071f79647d0a619a13ffb7ae739/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.281064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] RBAC: Authorizing `identity:create_grant(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd, user_id=47aae071f79647d0a619a13ffb7ae739, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.282673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.299209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 62/130] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:17 2022] PUT /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/47aae071f79647d0a619a13ffb7ae739/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:17.300411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.353684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:17.362132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ba33a-4c6f-4724-b8ec-0bc15f497e09 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.385156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:17.412697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.413721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.414434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.415663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.418849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.421448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.425186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 69/131] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:17 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:17.427192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.464600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528b32c-409d-4ce7-bc56-03001e4aef58 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.504740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.505679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.505679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.505922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.507070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.508557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.510441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 63/132] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:17 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:17.511046 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.563203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19986da-bf77-4cfc-9be4-be14fbd72fb1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.617082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.617969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.618321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.618658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/1842912f886f4b17b51cd35bebba1a7e/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.620841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] RBAC: Authorizing `identity:create_grant(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd, user_id=1842912f886f4b17b51cd35bebba1a7e, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.622665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.648416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 70/133] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:17 2022] PUT /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/1842912f886f4b17b51cd35bebba1a7e/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:17.649250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.698767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288bb3e4-b06f-472f-a33b-36845532b534 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.717976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:17.773257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.774551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.775353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.776219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] PATH_INFO: `/v3/projects/624580f535f14a4e883a2cedc84e9663` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.784258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] RBAC: Authorizing `identity:get_project(project_id=624580f535f14a4e883a2cedc84e9663)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.786197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.788807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 64/134] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:17 2022] GET /identity/v3/projects/624580f535f14a4e883a2cedc84e9663 => generated 312 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:17.800674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:17.807414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:17.890558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e347-0a88-4c60-a8dc-8cb83d13b133 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.930021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cedff-b01d-4f03-b44e-777fd065dba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:17.930668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cedff-b01d-4f03-b44e-777fd065dba8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.931054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cedff-b01d-4f03-b44e-777fd065dba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.931335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cedff-b01d-4f03-b44e-777fd065dba8 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.932701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cedff-b01d-4f03-b44e-777fd065dba8 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:17.934373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cedff-b01d-4f03-b44e-777fd065dba8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:17.936344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 71/135] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:17 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:17.937455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f6ed-3f1b-48df-bdfc-b1dbf9b01c40 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.937834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f6ed-3f1b-48df-bdfc-b1dbf9b01c40 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.938122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f6ed-3f1b-48df-bdfc-b1dbf9b01c40 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.939502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 72/136] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:17.944897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c912b9-a3d7-474d-ac71-3188260d96b2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:17.945298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c912b9-a3d7-474d-ac71-3188260d96b2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:17.945585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c912b9-a3d7-474d-ac71-3188260d96b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:17.963448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148ac38-2813-4661-a9d5-b8e23fd92a90 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:17.995853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b0c912b9-a3d7-474d-ac71-3188260d96b2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:17.997111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b0c912b9-a3d7-474d-ac71-3188260d96b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:18.011915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66358586-7684-42ec-af58-6177a311c568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:18.014952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66358586-7684-42ec-af58-6177a311c568 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:18.015234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66358586-7684-42ec-af58-6177a311c568 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:18.015505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66358586-7684-42ec-af58-6177a311c568 None admin] PATH_INFO: `/v3/projects/624580f535f14a4e883a2cedc84e9663/users/47aae071f79647d0a619a13ffb7ae739/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:18.019347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66358586-7684-42ec-af58-6177a311c568 None admin] RBAC: Authorizing `identity:create_grant(project_id=624580f535f14a4e883a2cedc84e9663, user_id=47aae071f79647d0a619a13ffb7ae739, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:18.020887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66358586-7684-42ec-af58-6177a311c568 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:18.048580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 65/137] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:17 2022] PUT /identity/v3/projects/624580f535f14a4e883a2cedc84e9663/users/47aae071f79647d0a619a13ffb7ae739/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:18.049414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66358586-7684-42ec-af58-6177a311c568 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:18.151550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 73/138] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:18.155019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:18.208903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66358586-7684-42ec-af58-6177a311c568 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:18.245523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf6e92-ad7f-4c18-a7c8-eff47cac8082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:18.246211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf6e92-ad7f-4c18-a7c8-eff47cac8082 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:18.246497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf6e92-ad7f-4c18-a7c8-eff47cac8082 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:18.246782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf6e92-ad7f-4c18-a7c8-eff47cac8082 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/47aae071f79647d0a619a13ffb7ae739/roles/66704401130047e78e714312f45defd9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:18.249119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf6e92-ad7f-4c18-a7c8-eff47cac8082 None admin] RBAC: Authorizing `identity:create_grant(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd, user_id=47aae071f79647d0a619a13ffb7ae739, role_id=66704401130047e78e714312f45defd9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:18.250686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf6e92-ad7f-4c18-a7c8-eff47cac8082 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:18.275617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 66/139] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:18 2022] PUT /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/47aae071f79647d0a619a13ffb7ae739/roles/66704401130047e78e714312f45defd9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:18.351744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db52a57-37e2-45b0-9442-1c102685f33b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:18.352055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db52a57-37e2-45b0-9442-1c102685f33b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:18.352334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db52a57-37e2-45b0-9442-1c102685f33b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:18.353833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 74/140] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:18.359460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb3f87-a36b-4685-9a85-122b2e3f715f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:18.359989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb3f87-a36b-4685-9a85-122b2e3f715f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:18.360411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb3f87-a36b-4685-9a85-122b2e3f715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:18.381044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-54fb3f87-a36b-4685-9a85-122b2e3f715f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:18.382555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-54fb3f87-a36b-4685-9a85-122b2e3f715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:18.458582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 67/141] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:18.685932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db52a57-37e2-45b0-9442-1c102685f33b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:18.708938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:18.786989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb3f87-a36b-4685-9a85-122b2e3f715f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:18.815920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db52a57-37e2-45b0-9442-1c102685f33b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:18.884555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:18.885125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:18.885408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:18.885678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:18.887015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:18.888299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:18.889913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 75/142] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:18.896739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:18.900429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:18.937280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb3f87-a36b-4685-9a85-122b2e3f715f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:18.972917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:18.974313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:18.974759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:18.975195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:18.989420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:18.995293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:18.996176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:18.996936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:19.001498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 68/143] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:58:19.008983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:19.160375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:19.190779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011dc07-633f-4c7c-a05b-6e08215a7d61 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:19.202647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3c4b8-2f16-477b-a254-4e22614b8abb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:19.212108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:19.248578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:19.250026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:19.250571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:19.251049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:19.252032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:19.252538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:19.254294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:19.271090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 69/144] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:19.277271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:19.296374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:19.296992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:19.297273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:19.297547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:19.298204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:19.299678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:19.344476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 76/145] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:58:18 2022] POST /identity/v3/users => generated 283 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:19.448734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fedd4c-f5b4-4f27-894b-43ec948c3bd2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:19.503325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:19.504011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:19.504297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:19.504576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] PATH_INFO: `/v3/users/b7122418202c4c39a6f4095fbcdb9ca0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:19.508441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] RBAC: Authorizing `identity:get_user(user_id=b7122418202c4c39a6f4095fbcdb9ca0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:19.509899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:19.511883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 70/146] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:19 2022] GET /identity/v3/users/b7122418202c4c39a6f4095fbcdb9ca0 => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:19.517829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:19.674580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0792fd-d50e-4385-8207-859860a3aecb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:19.718983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1c689-3a79-4d5e-88cf-e6f6870abb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:19.719545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1c689-3a79-4d5e-88cf-e6f6870abb66 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:19.719855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1c689-3a79-4d5e-88cf-e6f6870abb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:19.720134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1c689-3a79-4d5e-88cf-e6f6870abb66 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:19.721290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1c689-3a79-4d5e-88cf-e6f6870abb66 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:19.722756 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1c689-3a79-4d5e-88cf-e6f6870abb66 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:19.724752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 77/147] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:19 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:19.730528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:19.809714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:19.883630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f7ce0-5896-41ec-acea-ff2b618c8b96 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:19.924823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72f47c-3259-41bf-87f7-2b865421ea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:19.925501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72f47c-3259-41bf-87f7-2b865421ea76 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:19.925788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72f47c-3259-41bf-87f7-2b865421ea76 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:19.926075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72f47c-3259-41bf-87f7-2b865421ea76 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/b7122418202c4c39a6f4095fbcdb9ca0/roles/b6eb85a18e7a483d859c5d57f92888b8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:19.929695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72f47c-3259-41bf-87f7-2b865421ea76 None admin] RBAC: Authorizing `identity:create_grant(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd, user_id=b7122418202c4c39a6f4095fbcdb9ca0, role_id=b6eb85a18e7a483d859c5d57f92888b8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:19.931154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72f47c-3259-41bf-87f7-2b865421ea76 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:19.953727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 71/148] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:19 2022] PUT /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/users/b7122418202c4c39a6f4095fbcdb9ca0/roles/b6eb85a18e7a483d859c5d57f92888b8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:20.003259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb695-3ac4-4013-bff3-fdffa2aacb55 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:20.003259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb695-3ac4-4013-bff3-fdffa2aacb55 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:20.003259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb695-3ac4-4013-bff3-fdffa2aacb55 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:20.003259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 78/149] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:20.009338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57707dd-e53b-4c62-ac2f-947f3f81c08e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:20.009828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57707dd-e53b-4c62-ac2f-947f3f81c08e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:20.009902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57707dd-e53b-4c62-ac2f-947f3f81c08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:20.040989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a57707dd-e53b-4c62-ac2f-947f3f81c08e None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:20.042223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a57707dd-e53b-4c62-ac2f-947f3f81c08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:20.138313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 72/150] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:20.684968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb695-3ac4-4013-bff3-fdffa2aacb55 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:20.701646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:20.808946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb695-3ac4-4013-bff3-fdffa2aacb55 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:20.848196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0130c18-41eb-40e1-b936-4837697be748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:20.848768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0130c18-41eb-40e1-b936-4837697be748 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:20.849048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0130c18-41eb-40e1-b936-4837697be748 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:20.849322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0130c18-41eb-40e1-b936-4837697be748 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:20.864230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0130c18-41eb-40e1-b936-4837697be748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b0130c18-41eb-40e1-b936-4837697be748 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:20.869341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:20.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:20.870924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 79/151] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:20.876481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57707dd-e53b-4c62-ac2f-947f3f81c08e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:20.902085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff547-786f-4e05-b059-d67abdb96951 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:20.902387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff547-786f-4e05-b059-d67abdb96951 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:20.902643 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff547-786f-4e05-b059-d67abdb96951 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:20.904851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 80/152] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:20.911066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e41b4-6df3-429e-ba0f-872dac76b9e6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:20.911340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e41b4-6df3-429e-ba0f-872dac76b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:20.911619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e41b4-6df3-429e-ba0f-872dac76b9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:20.942707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-df8e41b4-6df3-429e-ba0f-872dac76b9e6 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:20.943954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-df8e41b4-6df3-429e-ba0f-872dac76b9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:20.949914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:21.039193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 81/153] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:21.040545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc41e90-2843-403e-b8ef-694d00366406 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:21.040828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc41e90-2843-403e-b8ef-694d00366406 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:21.041101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc41e90-2843-403e-b8ef-694d00366406 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:21.042498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 82/154] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:21.056070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b6ab0-be99-4572-8ad6-663397c11348 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:21.056365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b6ab0-be99-4572-8ad6-663397c11348 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:21.056628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b6ab0-be99-4572-8ad6-663397c11348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:21.068623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:21.081134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57707dd-e53b-4c62-ac2f-947f3f81c08e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:21.084356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-290b6ab0-be99-4572-8ad6-663397c11348 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:21.085658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-290b6ab0-be99-4572-8ad6-663397c11348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:21.195356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 83/155] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:21.198466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdf223-5491-4a61-a964-8d64bd05080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:21.199103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf223-5491-4a61-a964-8d64bd05080a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:21.199401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf223-5491-4a61-a964-8d64bd05080a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:21.199676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf223-5491-4a61-a964-8d64bd05080a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:21.200381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdf223-5491-4a61-a964-8d64bd05080a None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:21.200686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdf223-5491-4a61-a964-8d64bd05080a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:21.202611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdf223-5491-4a61-a964-8d64bd05080a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:21.230789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 73/156] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:21.236602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b6ab0-be99-4572-8ad6-663397c11348 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:21.418923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b6ab0-be99-4572-8ad6-663397c11348 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:21.456624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30aaa-58de-4115-b88d-98a059f940b5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:21.456906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30aaa-58de-4115-b88d-98a059f940b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:21.457097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30aaa-58de-4115-b88d-98a059f940b5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:21.458448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 74/157] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:21.467920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa16c-8883-4aef-bc77-d39aa82d6abc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:21.468255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa16c-8883-4aef-bc77-d39aa82d6abc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:21.468478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa16c-8883-4aef-bc77-d39aa82d6abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:21.472294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:21.488511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:21.489140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:21.489424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:21.489700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:21.496464 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:21.498469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:21.500464 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 84/158] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:21 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:21.523666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:21.534511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5e5fa16c-8883-4aef-bc77-d39aa82d6abc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:21.540574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5e5fa16c-8883-4aef-bc77-d39aa82d6abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:21.731448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf13173-bf77-4159-bfaa-7fc8dce3c550 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:21.772104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 75/159] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:21.773069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa16c-8883-4aef-bc77-d39aa82d6abc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:21.829316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:21.829937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:21.830219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:21.830494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:21.850771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:21.992362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:21.999957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:22.004005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:22.016241 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa16c-8883-4aef-bc77-d39aa82d6abc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:22.066895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:22.067549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:22.067854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:22.068191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:22.084834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:22.089915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:22.090745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:22.091562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 76/160] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:22.095947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:22.212728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 85/161] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:22.213659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:22.272278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:22.291226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2529d90-0a9c-4d9e-b7b6-f5b4ff9a40e2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:22.364880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:22.365537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:22.365814 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:22.366083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:22.397834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:22.402651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:22.403588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:22.404957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 77/162] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:22.411481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:22.416247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:22.422962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e192c-d1f1-477f-8b31-9ec299bd4d80 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:22.506416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:22.507046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:22.507331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:22.507604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:22.508313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:22.508619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:22.510015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:22.527079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 86/163] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:22.527969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:22.561490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe1ac-18a0-439c-a192-195055133f90 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:22.632490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:22.633152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:22.633439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:22.633722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:22.640453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:22.641933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:22.643741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 78/164] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:22.644381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:22.693194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:22.700748 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbeac8-9a41-4877-adc3-f11933bc1dd3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:22.755175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:22.755840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:22.756127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:22.756402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:22.779939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:22.785245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:22.786104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:22.787111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 87/165] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:22.792190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:22.832173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:22.889955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04f512-3daf-47ed-9355-fbeaacc77f73 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:22.916416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ac727-9c15-49ff-81e0-8673a47dc467 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:22.960711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:22.961368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:22.961651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:22.961924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] PATH_INFO: `/v3/users/1842912f886f4b17b51cd35bebba1a7e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:22.963074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] RBAC: Authorizing `identity:get_user(user_id=1842912f886f4b17b51cd35bebba1a7e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:22.964574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:22.970665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 88/166] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/users/1842912f886f4b17b51cd35bebba1a7e => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:22.971513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:22.972141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:22.972350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:22.972438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:22.972886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:22.973381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:22.973689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:22.975657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:22.995080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 79/167] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:22.996271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:23.134897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:23.150172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:23.153510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6248719-90fa-43bd-a710-355c86f783ce None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:23.200285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3243b43-1d7d-48bb-9ce2-a9b6a74625d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:23.221209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:23.222414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.222849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.223256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.224221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=47aae071f79647d0a619a13ffb7ae739) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:23.224680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:23.226418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:23.234656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 89/168] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/role_assignments?user.id=47aae071f79647d0a619a13ffb7ae739&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:23.235895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:23.260900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:23.261822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.262251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.262659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.263484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:23.264630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:23.266197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:23.282608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 80/169] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:23.283722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:23.347576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707c15c-eb71-4f9b-8f35-3b784c382933 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:23.392551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2eaf6-752c-4331-a898-268107f56927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:23.393106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2eaf6-752c-4331-a898-268107f56927 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.393391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2eaf6-752c-4331-a898-268107f56927 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.393677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2eaf6-752c-4331-a898-268107f56927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.461487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:23.482020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a211de-dea3-4f95-874f-3bbd578611b7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:23.528845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8715f-a1a9-41ed-a4c2-95ae7e3ed269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:23.529535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8715f-a1a9-41ed-a4c2-95ae7e3ed269 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.529794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8715f-a1a9-41ed-a4c2-95ae7e3ed269 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.530063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8715f-a1a9-41ed-a4c2-95ae7e3ed269 None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.531343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8715f-a1a9-41ed-a4c2-95ae7e3ed269 None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:23.532816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8715f-a1a9-41ed-a4c2-95ae7e3ed269 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:23.534566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 81/170] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:23 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:23.535509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6ae2b-6be9-41ff-8573-7277b4213d32 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.541195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6ae2b-6be9-41ff-8573-7277b4213d32 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.542275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6ae2b-6be9-41ff-8573-7277b4213d32 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.547253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 82/171] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:23 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:23.548006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6ae2b-6be9-41ff-8573-7277b4213d32 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:23.553583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2eaf6-752c-4331-a898-268107f56927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:23.555092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2eaf6-752c-4331-a898-268107f56927 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:23.568153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:23.693794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 90/172] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:23.695261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcebc8-af94-4fdf-963e-ac913611d9f5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.695549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcebc8-af94-4fdf-963e-ac913611d9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.695847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcebc8-af94-4fdf-963e-ac913611d9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.700436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6ae2b-6be9-41ff-8573-7277b4213d32 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:23.721110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c5bcebc8-af94-4fdf-963e-ac913611d9f5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:23.722287 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c5bcebc8-af94-4fdf-963e-ac913611d9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:23.744438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:23.745459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.745837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.746215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] PATH_INFO: `/v3/users/47aae071f79647d0a619a13ffb7ae739` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.747603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] RBAC: Authorizing `identity:get_user(user_id=47aae071f79647d0a619a13ffb7ae739)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:23.751346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:23.751346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 83/173] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:23 2022] GET /identity/v3/users/47aae071f79647d0a619a13ffb7ae739 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:23.752717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:23.790236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:23.813141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 91/174] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:23.820021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1156b-2fe0-4af2-8408-0b4ddf983e31 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.820507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1156b-2fe0-4af2-8408-0b4ddf983e31 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.821041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1156b-2fe0-4af2-8408-0b4ddf983e31 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.826428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 92/175] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:23.826428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1156b-2fe0-4af2-8408-0b4ddf983e31 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:23.856615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:23.890288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8b101-48bd-49cb-8f68-d1b400412888 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:23.935991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003cd56-f7fa-446a-9909-65758ddf1bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:23.936604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003cd56-f7fa-446a-9909-65758ddf1bfa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:23.936891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003cd56-f7fa-446a-9909-65758ddf1bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:23.937167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003cd56-f7fa-446a-9909-65758ddf1bfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:23.971796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1156b-2fe0-4af2-8408-0b4ddf983e31 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.017260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.017893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.018172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.018443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.019894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:24.021937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:24.023149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003cd56-f7fa-446a-9909-65758ddf1bfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:24.024092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 93/176] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:23 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:24.024648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.025121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003cd56-f7fa-446a-9909-65758ddf1bfa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:24.090938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.121720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.142502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b1957-e2db-4519-bff6-ec7903a5ec8a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.148960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 84/177] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:24.150185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fa9ae-38f9-4e51-af98-3cf1a5219138 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.150470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fa9ae-38f9-4e51-af98-3cf1a5219138 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.150989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fa9ae-38f9-4e51-af98-3cf1a5219138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.168657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-405fa9ae-38f9-4e51-af98-3cf1a5219138 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:24.169920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-405fa9ae-38f9-4e51-af98-3cf1a5219138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:24.181465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2e1e5-6d4d-4110-b788-f3dfa43b1f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.182066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2e1e5-6d4d-4110-b788-f3dfa43b1f92 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.182347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2e1e5-6d4d-4110-b788-f3dfa43b1f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.182620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2e1e5-6d4d-4110-b788-f3dfa43b1f92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.248160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 85/178] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:24.248160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e217388-5660-4c80-841d-11042d636ee9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.248160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e217388-5660-4c80-841d-11042d636ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.248160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e217388-5660-4c80-841d-11042d636ee9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.248160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 86/179] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:24.248160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e217388-5660-4c80-841d-11042d636ee9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.251198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2e1e5-6d4d-4110-b788-f3dfa43b1f92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:24.252697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2e1e5-6d4d-4110-b788-f3dfa43b1f92 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:24.306409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.346940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e217388-5660-4c80-841d-11042d636ee9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.350372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.363740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 94/180] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:24.363740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55470e-07d6-44ce-84c8-94730abef2ac None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.363740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55470e-07d6-44ce-84c8-94730abef2ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.363740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55470e-07d6-44ce-84c8-94730abef2ac None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.363740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 95/181] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:24.363740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55470e-07d6-44ce-84c8-94730abef2ac None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.390167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.390979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.391425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.391741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.392489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=1842912f886f4b17b51cd35bebba1a7e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:24.392793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:24.394209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:24.400158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 87/182] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/role_assignments?user.id=1842912f886f4b17b51cd35bebba1a7e&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:24.400752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.514502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55470e-07d6-44ce-84c8-94730abef2ac None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.524025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71927a64-56c7-4550-aef4-42d8e74f4b17 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.557622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.558270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.558544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.558821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.564244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ee7bc-a773-4bd7-81cc-801e2a4ef1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.564885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee7bc-a773-4bd7-81cc-801e2a4ef1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.565192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee7bc-a773-4bd7-81cc-801e2a4ef1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.565468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee7bc-a773-4bd7-81cc-801e2a4ef1e7 None admin] PATH_INFO: `/v3/projects/624580f535f14a4e883a2cedc84e9663` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.566707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ee7bc-a773-4bd7-81cc-801e2a4ef1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=624580f535f14a4e883a2cedc84e9663)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:24.568200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ee7bc-a773-4bd7-81cc-801e2a4ef1e7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:24.570101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 88/183] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/projects/624580f535f14a4e883a2cedc84e9663 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:24.571107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3712b-7715-46eb-8b5e-b83a47dbf97b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.571462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3712b-7715-46eb-8b5e-b83a47dbf97b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.571745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3712b-7715-46eb-8b5e-b83a47dbf97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.577949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:24.583201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:24.584169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 96/184] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:24.589912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.593973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-39c3712b-7715-46eb-8b5e-b83a47dbf97b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:24.595180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-39c3712b-7715-46eb-8b5e-b83a47dbf97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:24.612607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.631530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.672593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 89/185] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:24.681151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa26c6-94a8-4017-9505-c939f4c020ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.681442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa26c6-94a8-4017-9505-c939f4c020ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.681717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa26c6-94a8-4017-9505-c939f4c020ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.704832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-80fa26c6-94a8-4017-9505-c939f4c020ba None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:24.706255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80fa26c6-94a8-4017-9505-c939f4c020ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:24.709109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d6f37-8688-4e80-930f-da2e47d7736e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.743861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.744509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.744785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.745054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.746159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:24.747592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:24.754685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 97/186] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:24.754685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.780563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 90/187] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:24.781326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa26c6-94a8-4017-9505-c939f4c020ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.855171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.877192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324d2c2-0cc6-48c9-b55d-d0e2addaf61b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.902668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:24.912284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa26c6-94a8-4017-9505-c939f4c020ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:24.912983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.913611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.913886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.914153 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.925561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:24.930093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:24.931231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 98/188] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:24.937243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:24.959692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:24.960326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:24.960601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:24.960877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:24.961590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=624580f535f14a4e883a2cedc84e9663, user.id=47aae071f79647d0a619a13ffb7ae739) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:24.961891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:24.963304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:24.971383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 91/189] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/role_assignments?user.id=47aae071f79647d0a619a13ffb7ae739&scope.project.id=624580f535f14a4e883a2cedc84e9663&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:24.971383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.069261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28323a-a781-4711-81d3-4d6056f57d71 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.097302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31536e-2768-4800-8905-a64e533b628b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.107262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.108487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.108774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.109049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.109727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:25.110031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.111436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.120213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 99/190] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:25.120842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.125365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.142862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.143474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.143766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.144046 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.144776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] RBAC: Adding query filter params (role.id=66704401130047e78e714312f45defd9, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=47aae071f79647d0a619a13ffb7ae739) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:25.145076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.146492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.153060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 92/191] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:24 2022] GET /identity/v3/role_assignments?user.id=47aae071f79647d0a619a13ffb7ae739&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=66704401130047e78e714312f45defd9 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:25.153702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.189324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.256815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446272cb-b6ce-4311-9d5d-0fdc7db093ca None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.293320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5de08-f03f-4721-8e20-b240ba9bc7cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.295687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.296334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.296605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.296870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.297544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:25.297842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.299235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.309186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 100/192] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:25.309427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.335565 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.336168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.336449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.336721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.349886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:25.355261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:25.356179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:25.356946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 93/193] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:25.358218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.391994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.430900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20399e9f-781a-4fef-a44d-884c8992bef3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.443327 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.461594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56010e8-f4cc-4023-ac64-3a64210cebc8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.466039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.466672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.467468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.467743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.468900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.470349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.471948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 101/194] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:25.472712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.500206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.500830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.501111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.501405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] PATH_INFO: `/v3/users/b7122418202c4c39a6f4095fbcdb9ca0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.502517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] RBAC: Authorizing `identity:get_user(user_id=b7122418202c4c39a6f4095fbcdb9ca0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.503981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.505555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 94/195] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/users/b7122418202c4c39a6f4095fbcdb9ca0 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:25.506289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.572606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a74e-485b-42bb-9baf-a8081f69ee48 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.602292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.608106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.608505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31593a6-2625-45c8-ac80-9b9a06c6aff8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.608758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.609035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.609311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] PATH_INFO: `/v3/users/49dc53f8fefe4c60be6d9bf278c4670e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.623375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] RBAC: Authorizing `identity:get_user(user_id=49dc53f8fefe4c60be6d9bf278c4670e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.626096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.627979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 102/196] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/users/49dc53f8fefe4c60be6d9bf278c4670e => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:25.628638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.654136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.655273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.655547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.655859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.656521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:25.656818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.658194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.666272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 95/197] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:25.667047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.682271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.729778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6933c1-ac17-4fcd-adf6-1f8630be242a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.762114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.762723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.763006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.763282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.764463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.765084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef6c0-3cc1-4bfb-983a-685efee0a065 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.765908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.786579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 103/198] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:58:25 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:25.786579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:25.800976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.801615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.801902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.802173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:25.839882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.881622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:25.883056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:25.899710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:25.905169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8564-598a-4f5a-ad06-ba62630ac1a1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:25.952632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:25.953255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:25.953540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:25.953814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.010527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 96/199] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:26.011520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:26.038161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.040100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.103724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:26.135037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b134c3-a7c2-4200-9d4a-c1787797f009 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:26.153251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:26.169436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 104/200] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:26.170443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:26.181873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:26.182544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:26.183245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:26.183523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] PATH_INFO: `/v3/users/49dc53f8fefe4c60be6d9bf278c4670e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.185077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] RBAC: Authorizing `identity:get_user(user_id=49dc53f8fefe4c60be6d9bf278c4670e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.187908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.190032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 97/201] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/users/49dc53f8fefe4c60be6d9bf278c4670e => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:26.191195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:26.298333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cfbb7-5cd3-4297-a9fd-f58dd183c31b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:26.317965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df0385-3a15-467c-8a10-2f2305e5480e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:26.343694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:26.344552 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:26.344932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:26.345304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.346551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.348253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.349981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 105/202] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:26.350828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:26.359893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:26.360964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:26.361246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:26.361522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.363098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.365036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.367133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 98/203] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:26.367921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:26.368033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:26.440122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:26.463345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd9400-2488-41b9-bd6e-634c65fc2bf9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:26.501993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9c2b0-e00c-4e4a-9584-aa9759e70156 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:26.502256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:26.502764 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:26.503050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:26.503325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.547405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:26.548680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:26.549026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:26.549382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.550639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] RBAC: Adding query filter params (role.id=b6eb85a18e7a483d859c5d57f92888b8, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd, user.id=b7122418202c4c39a6f4095fbcdb9ca0) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:26.551018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.553267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.562235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 99/204] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/role_assignments?user.id=b7122418202c4c39a6f4095fbcdb9ca0&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=b6eb85a18e7a483d859c5d57f92888b8 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:26.563558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:26.592422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.595087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.617191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:26.703626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0767e-0b32-46ec-96d6-09ab202acfb6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:26.719732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:26.728830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 106/205] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:26.737581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:26.755378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:26.755378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:26.755378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:26.755378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.755837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=49dc53f8fefe4c60be6d9bf278c4670e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:26.755837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.755837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.766136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 100/206] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/role_assignments?user.id=49dc53f8fefe4c60be6d9bf278c4670e&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:26.877490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a39f1-0b3e-455d-9587-ecf5442801ed None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:26.896139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:26.921347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad60874-4096-4f00-99eb-55a07bc66d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:26.921994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60874-4096-4f00-99eb-55a07bc66d6a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:26.922276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60874-4096-4f00-99eb-55a07bc66d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:26.922617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60874-4096-4f00-99eb-55a07bc66d6a None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:26.923895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad60874-4096-4f00-99eb-55a07bc66d6a None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:26.925399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad60874-4096-4f00-99eb-55a07bc66d6a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:26.927402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 107/207] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:26.933346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:27.043980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3325a-0fa1-4775-84e6-45944046371f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:27.081843 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42b6c6-6714-47e0-9e5b-555b56681483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:27.082461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b6c6-6714-47e0-9e5b-555b56681483 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:27.082742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b6c6-6714-47e0-9e5b-555b56681483 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:27.083016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b6c6-6714-47e0-9e5b-555b56681483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:27.083765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42b6c6-6714-47e0-9e5b-555b56681483 None admin] RBAC: Adding query filter params (role.id=66704401130047e78e714312f45defd9, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=49dc53f8fefe4c60be6d9bf278c4670e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:27.084080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42b6c6-6714-47e0-9e5b-555b56681483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:27.085580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42b6c6-6714-47e0-9e5b-555b56681483 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:27.091453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 101/208] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:26 2022] GET /identity/v3/role_assignments?user.id=49dc53f8fefe4c60be6d9bf278c4670e&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=66704401130047e78e714312f45defd9 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:27.971540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85b75-337e-4e5b-812c-13762959f653 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:27.972090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85b75-337e-4e5b-812c-13762959f653 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:27.972090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85b75-337e-4e5b-812c-13762959f653 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:27.973655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 108/209] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:27.979232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db66cf-c197-4a47-ba02-68b5f3222599 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:27.979647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db66cf-c197-4a47-ba02-68b5f3222599 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:27.979888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db66cf-c197-4a47-ba02-68b5f3222599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:27.998985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-73db66cf-c197-4a47-ba02-68b5f3222599 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:28.000215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73db66cf-c197-4a47-ba02-68b5f3222599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:28.034744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99f382-9c2b-4468-8d8d-bd24ca5dc4c6 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.035042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99f382-9c2b-4468-8d8d-bd24ca5dc4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.035294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99f382-9c2b-4468-8d8d-bd24ca5dc4c6 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.036796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 109/210] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:28.038395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:28.042087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c65c-4ab2-407f-808d-6e8450cfb655 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.042439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c65c-4ab2-407f-808d-6e8450cfb655 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.042719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25c65c-4ab2-407f-808d-6e8450cfb655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.064202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9b25c65c-4ab2-407f-808d-6e8450cfb655 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:28.066058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9b25c65c-4ab2-407f-808d-6e8450cfb655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:28.072864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 102/211] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:28.136720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 110/212] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:28.352099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db66cf-c197-4a47-ba02-68b5f3222599 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:28.423690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c65c-4ab2-407f-808d-6e8450cfb655 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:28.454228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db66cf-c197-4a47-ba02-68b5f3222599 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:28.495776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:28.495776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.495776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.495776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.506037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:28.511281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:28.512384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:28.513174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 103/213] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:28.519357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:28.534463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25c65c-4ab2-407f-808d-6e8450cfb655 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:28.548616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:28.554966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:28.571141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:28.571800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.572086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.572356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.575120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:28.575120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:28.584717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 111/214] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:28.584717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:28.637916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4b4c9-39e4-48d6-bf69-c6e004b1ff8b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:28.674275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:28.675059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.675440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.675939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.676715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:28.677112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:28.678600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:28.684256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc844d-d894-4300-b9a2-db92dd31ee52 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:28.689064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 104/215] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:28.694347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:28.720450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cda412-0559-479c-8dad-6f40019e94f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:28.721065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cda412-0559-479c-8dad-6f40019e94f9 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.721387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cda412-0559-479c-8dad-6f40019e94f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.721668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cda412-0559-479c-8dad-6f40019e94f9 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.722371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cda412-0559-479c-8dad-6f40019e94f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:28.723863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cda412-0559-479c-8dad-6f40019e94f9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:28.750626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 112/216] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:58:28 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:58:28.799730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:28.806711 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10975c-2ba9-4813-855c-a88a204f7d7b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:28.829921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3136c-4057-4639-8c3b-be20d63a18eb None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.830226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3136c-4057-4639-8c3b-be20d63a18eb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.830480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3136c-4057-4639-8c3b-be20d63a18eb None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.832120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 113/217] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:28.837339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfac6ec-727e-46cd-85be-a54957b148d2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.837628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfac6ec-727e-46cd-85be-a54957b148d2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.837906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfac6ec-727e-46cd-85be-a54957b148d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.845359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:28.846014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:28.846295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:28.846567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] PATH_INFO: `/v3/users/7fe9c81085a943ae8130bd8da4b704ed` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:28.859406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1cfac6ec-727e-46cd-85be-a54957b148d2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:28.861154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1cfac6ec-727e-46cd-85be-a54957b148d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:28.861825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] RBAC: Authorizing `identity:get_user(user_id=7fe9c81085a943ae8130bd8da4b704ed)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:28.863379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:28.865116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 105/218] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:28 2022] GET /identity/v3/users/7fe9c81085a943ae8130bd8da4b704ed => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:28.871636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:28.896443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:28.951821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 114/219] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:28.991210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379406ed-97f9-4aff-9e4e-ed7e7b5d2f00 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.027522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.028579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.029075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.030074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.049214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1e9a-905e-4289-bf60-6681e27fec88 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.049706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1e9a-905e-4289-bf60-6681e27fec88 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.049787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1e9a-905e-4289-bf60-6681e27fec88 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.051349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 115/220] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:29.055819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:29.057694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba04e55-6dbd-49c1-a754-56d0416ed8dc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.058455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba04e55-6dbd-49c1-a754-56d0416ed8dc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.058869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba04e55-6dbd-49c1-a754-56d0416ed8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.081809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9ba04e55-6dbd-49c1-a754-56d0416ed8dc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:29.084281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9ba04e55-6dbd-49c1-a754-56d0416ed8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:29.113809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:29.115271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:29.185818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 116/221] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:29.238458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 106/222] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 00:58:29.239141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba04e55-6dbd-49c1-a754-56d0416ed8dc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:29.248958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:29.313341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:29.383172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:29.390040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae9045-ee33-48fe-bad2-94c7ad73e77d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.395302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba04e55-6dbd-49c1-a754-56d0416ed8dc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.463216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.463797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.464080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.464355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.465923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:29.468827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:29.471185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.471934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.472218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.472491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.473447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 107/223] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:29 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:29.476801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:29.487141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:29.495407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:29.496451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 117/224] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:29.497714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:29.596326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee09e4-3ea5-425c-95f2-b863aa07a41c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.611013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e1ce7-daaa-4b66-9e62-c6ac6653c363 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.619866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:29.636213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.637179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.637657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.638145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.646616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.647363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.647539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.647790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.648528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=7fe9c81085a943ae8130bd8da4b704ed) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:29.648840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:29.650292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:29.651205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:29.659808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 118/225] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:29 2022] GET /identity/v3/role_assignments?user.id=7fe9c81085a943ae8130bd8da4b704ed&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:29.659808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:29.665853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:29.666853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:29.667959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 108/226] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:29.676826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:29.708589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:29.769736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11da6e-92dd-4e97-8aaa-eb70508d67bb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.808878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.810178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.811438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.811941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.812964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:29.813462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:29.815040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:29.825057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 119/227] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:29.831356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:29.918334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08977bf5-81a6-4793-8bc4-418bf8737a1e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.937799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:29.955700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d9ca0-e22a-46ce-95c3-9c98d2fbad01 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:29.993623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.995029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.995850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:29.996675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.997455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:29.997908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] PATH_INFO: `/v3/users/49dc53f8fefe4c60be6d9bf278c4670e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:29.999440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:29.999916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:30.000780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:30.001563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] RBAC: Authorizing `identity:get_user(user_id=49dc53f8fefe4c60be6d9bf278c4670e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:30.002053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:30.003676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:30.005237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:30.008301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 120/228] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:29 2022] GET /identity/v3/users/49dc53f8fefe4c60be6d9bf278c4670e => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:30.014965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:30.022363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 109/229] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:30.027680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:30.078838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:30.160600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972bf7-b913-42ba-936d-5423de91e626 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:30.199907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:30.200850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:30.201354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:30.201808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] PATH_INFO: `/v3/users/49dc53f8fefe4c60be6d9bf278c4670e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:30.203557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] RBAC: Authorizing `identity:get_user(user_id=49dc53f8fefe4c60be6d9bf278c4670e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:30.205231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:30.207836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 110/230] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:30 2022] GET /identity/v3/users/49dc53f8fefe4c60be6d9bf278c4670e => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:30.216824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:30.235901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fdffc-1016-4a34-a315-315357beacdb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:30.278285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:30.278861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:30.279146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:30.279474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:30.280749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:30.282329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:30.284099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 121/231] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:30 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:30.290220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:30.313532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:30.410795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3aa98-c157-48aa-9c5c-21609ccd96eb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:30.432704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:30.444959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2bbf-3528-4355-873b-8fa486e8b358 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:30.479570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:30.480233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:30.480513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:30.480786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:30.484096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:30.485543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:30.496129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 111/232] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:30 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:30.506511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:30.521293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a005ecb-499c-4fd8-a72b-713f12f5b767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:30.521968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005ecb-499c-4fd8-a72b-713f12f5b767 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:30.522251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005ecb-499c-4fd8-a72b-713f12f5b767 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:30.522525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005ecb-499c-4fd8-a72b-713f12f5b767 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/49dc53f8fefe4c60be6d9bf278c4670e/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:30.525811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a005ecb-499c-4fd8-a72b-713f12f5b767 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb680eb41a4a40bcb931685c4b20c1b5, user_id=49dc53f8fefe4c60be6d9bf278c4670e, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:30.527266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a005ecb-499c-4fd8-a72b-713f12f5b767 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:30.551131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 122/233] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:30 2022] PUT /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/49dc53f8fefe4c60be6d9bf278c4670e/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:30.663169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c09f47-fa4a-4a4c-9fe0-6da80495b67d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:30.705337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4c93d-cd51-4491-a59b-4a3583005f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:30.706011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c93d-cd51-4491-a59b-4a3583005f06 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:30.706290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c93d-cd51-4491-a59b-4a3583005f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:30.706564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4c93d-cd51-4491-a59b-4a3583005f06 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/49dc53f8fefe4c60be6d9bf278c4670e/roles/66704401130047e78e714312f45defd9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:30.710340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c93d-cd51-4491-a59b-4a3583005f06 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb680eb41a4a40bcb931685c4b20c1b5, user_id=49dc53f8fefe4c60be6d9bf278c4670e, role_id=66704401130047e78e714312f45defd9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:30.711885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4c93d-cd51-4491-a59b-4a3583005f06 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:30.738235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 112/234] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:30 2022] PUT /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/49dc53f8fefe4c60be6d9bf278c4670e/roles/66704401130047e78e714312f45defd9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 00:58:31.216280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01e14f-b4a7-4d24-9dc0-0b72de234bcd None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:31.217024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01e14f-b4a7-4d24-9dc0-0b72de234bcd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:31.217497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01e14f-b4a7-4d24-9dc0-0b72de234bcd None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:31.219057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 123/235] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:31.224400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98ebee-0d72-42d1-b891-48dbdf6319af None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:31.224859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98ebee-0d72-42d1-b891-48dbdf6319af None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:31.224936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98ebee-0d72-42d1-b891-48dbdf6319af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:31.229688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b667-5719-4a06-ba7f-d06bfad7b863 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:31.229688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b667-5719-4a06-ba7f-d06bfad7b863 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:31.229688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b667-5719-4a06-ba7f-d06bfad7b863 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:31.230261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 124/236] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:31.235498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841b849-ec1a-42af-bd7f-b99181bd3255 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:31.236101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841b849-ec1a-42af-bd7f-b99181bd3255 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:31.236999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841b849-ec1a-42af-bd7f-b99181bd3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:31.249981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1a98ebee-0d72-42d1-b891-48dbdf6319af None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:31.251195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1a98ebee-0d72-42d1-b891-48dbdf6319af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:31.272841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8841b849-ec1a-42af-bd7f-b99181bd3255 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:31.274036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8841b849-ec1a-42af-bd7f-b99181bd3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:31.318618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:31.379105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 113/237] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:31.417492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 125/238] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:31.693421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98ebee-0d72-42d1-b891-48dbdf6319af None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:31.800242 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98ebee-0d72-42d1-b891-48dbdf6319af None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:31.820866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841b849-ec1a-42af-bd7f-b99181bd3255 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:31.833005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:31.853910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:31.854563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:31.854846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:31.855118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:31.875118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:31.880119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:31.880962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:31.881739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 114/239] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:31.887080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:31.941994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:31.983784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841b849-ec1a-42af-bd7f-b99181bd3255 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.031079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf78aa-9de8-4125-9d1e-93a42be73dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.033304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf78aa-9de8-4125-9d1e-93a42be73dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.033765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf78aa-9de8-4125-9d1e-93a42be73dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.034190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf78aa-9de8-4125-9d1e-93a42be73dd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.036306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf78aa-9de8-4125-9d1e-93a42be73dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:32.038308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf78aa-9de8-4125-9d1e-93a42be73dd3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:32.042551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 126/240] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:32.042668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b923d5-e5e7-4f7f-b68a-7ee517363884 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.044097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf475ff-4b21-4fed-ab2d-7ee1daa6e698 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.044584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf475ff-4b21-4fed-ab2d-7ee1daa6e698 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.045733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf475ff-4b21-4fed-ab2d-7ee1daa6e698 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.049555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 127/241] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:32.055038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c816-820c-4088-8e47-0536aa77397f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.055576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c816-820c-4088-8e47-0536aa77397f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.056073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7c816-820c-4088-8e47-0536aa77397f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.075304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7bc7c816-820c-4088-8e47-0536aa77397f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:32.077360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7bc7c816-820c-4088-8e47-0536aa77397f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:32.086017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.086699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.086980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.087252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.087975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:32.088276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:32.089675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:32.100019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 115/242] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:32.100908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:32.151378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 128/243] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:32.152282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c816-820c-4088-8e47-0536aa77397f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:32.162244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:32.204103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:32.219029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b5d26-6dfa-4582-8aef-1510a717933b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.254641 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.255279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.255510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.255792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.256172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7c816-820c-4088-8e47-0536aa77397f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.256543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:32.257985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:32.290791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 116/244] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:58:32 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:32.297627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.298406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.298687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.298958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] PATH_INFO: `/v3/users/cf033e89332647bcab0470937d465a74` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.329239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cf033e89332647bcab0470937d465a74)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:32.330823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:32.332892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 129/245] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:32 2022] GET /identity/v3/users/cf033e89332647bcab0470937d465a74 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:32.338644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:32.463151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf06830-3e31-4e43-91d5-ceb7d3545ed7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.499404 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.500061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.500398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.500669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.515313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:32.518050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:32.591837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:32.596599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:32.598264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:32.638760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7fa56e-bd61-42a6-89a4-ea5a37a620c4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.679294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.680229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.680703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.681071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.693725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:32.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:32.699928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:32.700711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:32.701930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 130/246] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:32.707849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:32.721120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 117/247] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:32.728819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:32.798886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:32.848086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6644400-90e9-4216-bf8e-1fd695847f2b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.856697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582be05-c941-4c85-9b8a-c15e7c57346d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:32.890244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:32.906826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567425-5367-4807-a701-d76370a82a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.907432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567425-5367-4807-a701-d76370a82a4e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.907711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567425-5367-4807-a701-d76370a82a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.908048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567425-5367-4807-a701-d76370a82a4e None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.908147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:32.908717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567425-5367-4807-a701-d76370a82a4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:32.908817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:32.909018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567425-5367-4807-a701-d76370a82a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:32.909124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:32.909296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:32.910407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567425-5367-4807-a701-d76370a82a4e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:32.910513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:32.912125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:32.914387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 118/248] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:32 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:32.920909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:32.924640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 131/249] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:32.942049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567425-5367-4807-a701-d76370a82a4e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:33.059380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0708e-16c1-4043-964e-4c3f815623e8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:33.077643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:33.082820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567425-5367-4807-a701-d76370a82a4e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:33.106998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1849ae-62b7-4945-a80a-afb28b79f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:33.108611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1849ae-62b7-4945-a80a-afb28b79f46f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.108892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1849ae-62b7-4945-a80a-afb28b79f46f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.109167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1849ae-62b7-4945-a80a-afb28b79f46f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.109902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1849ae-62b7-4945-a80a-afb28b79f46f None admin] RBAC: Adding query filter params (role.id=b6eb85a18e7a483d859c5d57f92888b8, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=cf033e89332647bcab0470937d465a74) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:33.110210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1849ae-62b7-4945-a80a-afb28b79f46f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:33.112486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1849ae-62b7-4945-a80a-afb28b79f46f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:33.118774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 119/250] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:32 2022] GET /identity/v3/role_assignments?user.id=cf033e89332647bcab0470937d465a74&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=b6eb85a18e7a483d859c5d57f92888b8 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:33.121020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74ea00-bdff-45e0-8cd8-0e5f5e5ffd1e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.121543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74ea00-bdff-45e0-8cd8-0e5f5e5ffd1e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.122020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74ea00-bdff-45e0-8cd8-0e5f5e5ffd1e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.124248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 120/251] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:33.124366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:33.124949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.125225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.125508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] PATH_INFO: `/v3/users/7fe9c81085a943ae8130bd8da4b704ed` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.126714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] RBAC: Authorizing `identity:get_user(user_id=7fe9c81085a943ae8130bd8da4b704ed)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:33.131792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:33.131792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 132/252] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:32 2022] GET /identity/v3/users/7fe9c81085a943ae8130bd8da4b704ed => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:33.133768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c53054-e1bc-4110-8822-bf93670ea7d2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.134305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c53054-e1bc-4110-8822-bf93670ea7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.134830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c53054-e1bc-4110-8822-bf93670ea7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.136033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:33.170831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-74c53054-e1bc-4110-8822-bf93670ea7d2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:33.172212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74c53054-e1bc-4110-8822-bf93670ea7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:33.208753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:33.264952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8345-f27a-4176-8d31-2e47a2daf7e1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:33.307181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:33.308774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.309227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.309666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.311542 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:33.313856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:33.316447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 133/253] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:33 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:33.319515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 121/254] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:33.322671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:33.403553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca560a-903c-483d-b82f-74324d54c6c2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.403803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca560a-903c-483d-b82f-74324d54c6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.404217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca560a-903c-483d-b82f-74324d54c6c2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.406809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 122/255] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:33.415384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e61d4f-b6f4-4822-a174-ed7f513c9f3c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.416038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e61d4f-b6f4-4822-a174-ed7f513c9f3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.416354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e61d4f-b6f4-4822-a174-ed7f513c9f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.443779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156bfbd-50d6-4379-b1c7-f2c191d4490f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:33.454592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-44e61d4f-b6f4-4822-a174-ed7f513c9f3c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:33.456127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44e61d4f-b6f4-4822-a174-ed7f513c9f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:33.467789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:33.482254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:33.483499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:33.483796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:33.484072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/7fe9c81085a943ae8130bd8da4b704ed/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:33.489174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb680eb41a4a40bcb931685c4b20c1b5, user_id=7fe9c81085a943ae8130bd8da4b704ed, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:33.490642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:33.519214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 134/256] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:33 2022] PUT /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/7fe9c81085a943ae8130bd8da4b704ed/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 00:58:33.581689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 123/257] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:33.880497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.007774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c0bbf-5347-4144-a5f4-99119cfc0733 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:34.055869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:34.056508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:34.056792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:34.057068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:34.067670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:34.073373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:34.074272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:34.075065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 135/258] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:34.082605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e61d4f-b6f4-4822-a174-ed7f513c9f3c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.134303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:34.138147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.217594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:34.251577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e61d4f-b6f4-4822-a174-ed7f513c9f3c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:34.296383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:34.296994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:34.297275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:34.297546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:34.298210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:34.298516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:34.300135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:34.316768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ccc34-a649-418e-885f-f1d5e5934a70 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:34.320232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 124/259] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:34.325631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.392167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:34.392748 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:34.393092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:34.393365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:34.416131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:34.426196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:34.427130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:34.429639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 136/260] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:58:34.437890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.532979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:34.540054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06f7b5-a05d-4972-83d8-a175ae1633dc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:34.616626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341ce70-17cb-471a-a8ee-ae05287bbde1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:34.625905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:34.626558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:34.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:34.627117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] PATH_INFO: `/v3/users/49dc53f8fefe4c60be6d9bf278c4670e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:34.627595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:34.632439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] RBAC: Authorizing `identity:get_user(user_id=49dc53f8fefe4c60be6d9bf278c4670e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:34.633897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:34.635504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 125/261] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:34 2022] GET /identity/v3/users/49dc53f8fefe4c60be6d9bf278c4670e => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:34.644758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.684466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:34.685093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:34.685387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:34.685660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:34.686977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:34.687284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:34.688782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:34.701985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 137/262] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:34.707334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.768951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dcd0f-b68c-40d8-8412-63acd6d9a5f1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:34.832509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:34.833162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:34.833439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:34.833723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:34.876424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f25496-6cc0-4f5c-9eab-27f6d8cd37f2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:34.905540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:34.937380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:34.938030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:34.938314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:34.938594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] PATH_INFO: `/v3/users/49dc53f8fefe4c60be6d9bf278c4670e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:34.944446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] RBAC: Authorizing `identity:get_user(user_id=49dc53f8fefe4c60be6d9bf278c4670e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:34.945929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:34.947715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 138/263] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:34 2022] GET /identity/v3/users/49dc53f8fefe4c60be6d9bf278c4670e => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:34.952862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:34.953088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:34.954337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:35.005641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:35.115459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78e194-2c8a-4e93-bbae-aca3692bd8cd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:35.165432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:35.165967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.166247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.166520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.219171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 126/264] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:35.224523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:35.275286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:35.277176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:35.327561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:35.397319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:35.459659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c4c86-218d-42ad-9dc8-1c16060ca7f8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:35.460920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 139/265] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 00:58:35.466156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:35.520479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:35.521132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.521414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.521692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.522879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:35.524352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:35.526342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 127/266] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:35 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:35.532628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:35.696724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a0c7e-4611-4c69-8132-c91705e1d3fc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:35.700781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc137ebf-2054-4696-8805-76a004e35c60 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:35.730855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:35.739684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f260c5-8253-4dc7-8bf4-6034edbf18b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:35.740295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f260c5-8253-4dc7-8bf4-6034edbf18b3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.740581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f260c5-8253-4dc7-8bf4-6034edbf18b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.740856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f260c5-8253-4dc7-8bf4-6034edbf18b3 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.742961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f260c5-8253-4dc7-8bf4-6034edbf18b3 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:35.744467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f260c5-8253-4dc7-8bf4-6034edbf18b3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:35.747775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 140/267] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:35 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:35.749444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c563de7-2083-42a3-b85f-ad18ab58e0a2 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.749780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c563de7-2083-42a3-b85f-ad18ab58e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.750022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c563de7-2083-42a3-b85f-ad18ab58e0a2 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.751296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 141/268] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:35.757375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c563de7-2083-42a3-b85f-ad18ab58e0a2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:35.780870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0dc8-cb28-4697-b445-a6d3de909181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:35.781497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0dc8-cb28-4697-b445-a6d3de909181 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.782089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0dc8-cb28-4697-b445-a6d3de909181 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.782373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0dc8-cb28-4697-b445-a6d3de909181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.783399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0dc8-cb28-4697-b445-a6d3de909181 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=49dc53f8fefe4c60be6d9bf278c4670e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:35.783711 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0dc8-cb28-4697-b445-a6d3de909181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:35.785688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0dc8-cb28-4697-b445-a6d3de909181 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:35.789735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:35.794279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 128/269] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:35 2022] GET /identity/v3/role_assignments?user.id=49dc53f8fefe4c60be6d9bf278c4670e&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:35.795885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca8572-7dca-4350-9a51-7f4b16fce5dd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.796361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca8572-7dca-4350-9a51-7f4b16fce5dd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.796840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca8572-7dca-4350-9a51-7f4b16fce5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.835257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ffca8572-7dca-4350-9a51-7f4b16fce5dd None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:35.836521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ffca8572-7dca-4350-9a51-7f4b16fce5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:35.909299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c563de7-2083-42a3-b85f-ad18ab58e0a2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:35.927524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 129/270] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:35.929196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2c065-863c-4dbe-8481-a85dc8328ee4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.929689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2c065-863c-4dbe-8481-a85dc8328ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.930157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2c065-863c-4dbe-8481-a85dc8328ee4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.931717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 130/271] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:35.933049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c7f-acf5-4548-9dec-371a215de438 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.933617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c7f-acf5-4548-9dec-371a215de438 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.934101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c7f-acf5-4548-9dec-371a215de438 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.935568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 131/272] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:35.941284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2126e39-67e8-4171-9d4a-2c77de542b9b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.941810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2126e39-67e8-4171-9d4a-2c77de542b9b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.942298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2126e39-67e8-4171-9d4a-2c77de542b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.954582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e166e-a096-4c8f-9896-67bd503fccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:35.955195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e166e-a096-4c8f-9896-67bd503fccc9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.955474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e166e-a096-4c8f-9896-67bd503fccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.955761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e166e-a096-4c8f-9896-67bd503fccc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:35.956550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e166e-a096-4c8f-9896-67bd503fccc9 None admin] RBAC: Adding query filter params (role.id=66704401130047e78e714312f45defd9, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=49dc53f8fefe4c60be6d9bf278c4670e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:35.956862 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e166e-a096-4c8f-9896-67bd503fccc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:35.958329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e166e-a096-4c8f-9896-67bd503fccc9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:35.963355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d2126e39-67e8-4171-9d4a-2c77de542b9b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:35.964613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 142/273] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:35 2022] GET /identity/v3/role_assignments?user.id=49dc53f8fefe4c60be6d9bf278c4670e&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=66704401130047e78e714312f45defd9 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:35.965154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d2126e39-67e8-4171-9d4a-2c77de542b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:35.965807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db7d96-cfa8-42a5-9712-ac8aa8c39394 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:35.966091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db7d96-cfa8-42a5-9712-ac8aa8c39394 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:35.966360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db7d96-cfa8-42a5-9712-ac8aa8c39394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.012955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-07db7d96-cfa8-42a5-9712-ac8aa8c39394 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:36.014855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-07db7d96-cfa8-42a5-9712-ac8aa8c39394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:36.043980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 132/274] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:36.129660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:36.141165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 143/275] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:36.251378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2126e39-67e8-4171-9d4a-2c77de542b9b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.317073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:36.361076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2126e39-67e8-4171-9d4a-2c77de542b9b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:36.370097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db7d96-cfa8-42a5-9712-ac8aa8c39394 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.403975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:36.405186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.405542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.405815 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.417178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:36.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:36.428707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:36.429491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 133/276] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:36.433965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.507360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db7d96-cfa8-42a5-9712-ac8aa8c39394 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:36.543414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:36.544072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.544354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.544626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:36.547703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:36.549439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 144/277] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:36.550121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.554867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf79fc7-8a32-49ef-b6bc-2fad18713812 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:36.574933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:36.589025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc2f0b-51a1-4de2-8d97-1f1770899a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:36.589627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc2f0b-51a1-4de2-8d97-1f1770899a52 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.589904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc2f0b-51a1-4de2-8d97-1f1770899a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.590169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc2f0b-51a1-4de2-8d97-1f1770899a52 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.590819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc2f0b-51a1-4de2-8d97-1f1770899a52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:36.591115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc2f0b-51a1-4de2-8d97-1f1770899a52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:36.593031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc2f0b-51a1-4de2-8d97-1f1770899a52 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:36.600928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 134/278] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:36.602252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512baea2-cf3e-4dda-b221-044cb256f5d5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.602531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512baea2-cf3e-4dda-b221-044cb256f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.602798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512baea2-cf3e-4dda-b221-044cb256f5d5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.604036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 135/279] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:36.604685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512baea2-cf3e-4dda-b221-044cb256f5d5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.619025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:36.656454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1dd50-5611-4d0c-badd-20e491291a6b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:36.689579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:36.690204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.690482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.690758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.700461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:36.703773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512baea2-cf3e-4dda-b221-044cb256f5d5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:36.705396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:36.706452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 145/280] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:36.707240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.741862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8852f5-b4e7-444c-b3b1-ca448df459c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:36.742449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8852f5-b4e7-444c-b3b1-ca448df459c6 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.742722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8852f5-b4e7-444c-b3b1-ca448df459c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.742991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8852f5-b4e7-444c-b3b1-ca448df459c6 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.743621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8852f5-b4e7-444c-b3b1-ca448df459c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:36.745087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8852f5-b4e7-444c-b3b1-ca448df459c6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:36.766312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 136/281] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:58:36 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:36.766312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480693-20ee-4548-a344-251119e69488 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.768043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480693-20ee-4548-a344-251119e69488 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.768257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b480693-20ee-4548-a344-251119e69488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.796143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6b480693-20ee-4548-a344-251119e69488 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:36.797393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6b480693-20ee-4548-a344-251119e69488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:36.817726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591ec10-2872-4366-b350-55c212d05fec None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:36.830039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:36.848691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:36.860697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:36.861347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:36.861628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:36.861904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] PATH_INFO: `/v3/users/cf033e89332647bcab0470937d465a74` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:36.863570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] RBAC: Authorizing `identity:get_user(user_id=cf033e89332647bcab0470937d465a74)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:36.865094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:36.871178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 146/282] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:36 2022] GET /identity/v3/users/cf033e89332647bcab0470937d465a74 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:36.871178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.891795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 137/283] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:36.891795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b480693-20ee-4548-a344-251119e69488 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:36.986730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693816-9c33-4cd8-a667-119d13d12a97 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.020436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b480693-20ee-4548-a344-251119e69488 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.043235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.044182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.044642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.044910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.045616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:37.045917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:37.047283 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:37.057257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 147/284] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:37.060009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.060785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.061074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.061340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.062549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:37.062886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:37.064027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:37.065695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 138/285] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:36 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:37.072745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:37.118780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:37.127386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:37.192918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc153d-a674-4298-9a82-731fa88813c8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.199976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb66eeb-a508-4d62-a111-4ba063551918 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.228618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.229257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.229539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.229816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] PATH_INFO: `/v3/users/1fe8f73f42c7455aba309dcab23888cd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.242988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.243787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.244077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.244350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/cf033e89332647bcab0470937d465a74/roles/b6eb85a18e7a483d859c5d57f92888b8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.244457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] RBAC: Authorizing `identity:get_user(user_id=1fe8f73f42c7455aba309dcab23888cd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:37.246178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:37.246294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] RBAC: Authorizing `identity:create_grant(project_id=fb680eb41a4a40bcb931685c4b20c1b5, user_id=cf033e89332647bcab0470937d465a74, role_id=b6eb85a18e7a483d859c5d57f92888b8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:37.247683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:37.248958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 148/286] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:37 2022] GET /identity/v3/users/1fe8f73f42c7455aba309dcab23888cd => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:37.249929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:37.269945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 139/287] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:58:37 2022] PUT /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/users/cf033e89332647bcab0470937d465a74/roles/b6eb85a18e7a483d859c5d57f92888b8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:37.271852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:37.393682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e314f98-631f-4402-ae1d-b4572fa1b87e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.403787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76853f20-5b49-4ad8-aaae-0a9c68ff1fec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.421084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:37.447798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.447798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.447798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.447798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.449196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, user.id=1fe8f73f42c7455aba309dcab23888cd, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:37.449196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:37.449992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:37.454351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:37.458197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 140/288] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 6 00:58:37 2022] GET /identity/v3/role_assignments?user.id=1fe8f73f42c7455aba309dcab23888cd&scope.system=all&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:37.485183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.485919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.486241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.486523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.501809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:37.506847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:37.507723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:37.508546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 149/289] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:37.514523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:37.634647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ef3c8-bbcb-4448-a9d3-c6fe7771a3c1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.669604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.670663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.670953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.671223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.671916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:37.672216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:37.673596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:37.682402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 141/290] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:37.688668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:37.797755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374e713-4823-4081-862f-c28c82ad95dc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:37.832672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:37.833302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:37.833581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:37.833854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] PATH_INFO: `/v3/users/7fe9c81085a943ae8130bd8da4b704ed` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:37.834967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] RBAC: Authorizing `identity:get_user(user_id=7fe9c81085a943ae8130bd8da4b704ed)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:37.836496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:37.838364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 150/291] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:37 2022] GET /identity/v3/users/7fe9c81085a943ae8130bd8da4b704ed => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:37.844808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:37.879796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:37.969069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a05f6e-c0d7-4d7d-bc63-31690f909a0d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:38.025467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:38.026092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:38.026369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:38.026641 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:38.103375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:38.104930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:38.130940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:38.208745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 142/292] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:38.214337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:38.266981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:38.350350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9c83c-5122-4f36-92f9-f487324137b8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:38.397317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf2ff0-e7ef-4b0d-ade1-3c94fc65e223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:38.398017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2ff0-e7ef-4b0d-ade1-3c94fc65e223 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:38.398231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2ff0-e7ef-4b0d-ade1-3c94fc65e223 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:38.398511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2ff0-e7ef-4b0d-ade1-3c94fc65e223 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:38.399653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf2ff0-e7ef-4b0d-ade1-3c94fc65e223 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:38.401111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf2ff0-e7ef-4b0d-ade1-3c94fc65e223 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:38.402901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 151/293] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:38 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:38.409336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:38.534256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a581c2-89a1-4f39-ab3d-f74f5c04f2c2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:38.557026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:38.569105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39137b2-a563-4712-a5cc-dfcd211046d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:38.569715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39137b2-a563-4712-a5cc-dfcd211046d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:38.569994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39137b2-a563-4712-a5cc-dfcd211046d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:38.570270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39137b2-a563-4712-a5cc-dfcd211046d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:38.571056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39137b2-a563-4712-a5cc-dfcd211046d1 None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=7fe9c81085a943ae8130bd8da4b704ed) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:38.571366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39137b2-a563-4712-a5cc-dfcd211046d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:38.572839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39137b2-a563-4712-a5cc-dfcd211046d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:38.580141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 143/294] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:38 2022] GET /identity/v3/role_assignments?user.id=7fe9c81085a943ae8130bd8da4b704ed&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:39.180555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f2211-948b-4446-ba74-a1880b38a606 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.180835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f2211-948b-4446-ba74-a1880b38a606 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.181345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f2211-948b-4446-ba74-a1880b38a606 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.183057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 152/295] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:39.189281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd8838-dc07-42c6-9801-80206419e196 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.189642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd8838-dc07-42c6-9801-80206419e196 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.189921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd8838-dc07-42c6-9801-80206419e196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.211238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bfbd8838-dc07-42c6-9801-80206419e196 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:39.212460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bfbd8838-dc07-42c6-9801-80206419e196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:39.288513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 144/296] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:39.419385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f34da-ee66-413f-a26e-b404edbb5da9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.420975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f34da-ee66-413f-a26e-b404edbb5da9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.421471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f34da-ee66-413f-a26e-b404edbb5da9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.423311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 153/297] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:39.428682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c913cc-6522-4626-9ffb-e9210a91deaf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.429209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c913cc-6522-4626-9ffb-e9210a91deaf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.429681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c913cc-6522-4626-9ffb-e9210a91deaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.448167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-84c913cc-6522-4626-9ffb-e9210a91deaf None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:39.449636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84c913cc-6522-4626-9ffb-e9210a91deaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:39.517921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 145/298] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:39.582321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f34da-ee66-413f-a26e-b404edbb5da9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:39.689336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f34da-ee66-413f-a26e-b404edbb5da9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:39.696571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:39.707823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86c67-3a33-417d-b121-263f76d44480 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.708704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86c67-3a33-417d-b121-263f76d44480 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.709174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86c67-3a33-417d-b121-263f76d44480 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.710709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 146/299] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:39.717756 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5358-cf06-4956-ad9f-8d4ae76eca52 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.718043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5358-cf06-4956-ad9f-8d4ae76eca52 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.718465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5358-cf06-4956-ad9f-8d4ae76eca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.727248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:39.728487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.728769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.729038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.741173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-773b5358-cf06-4956-ad9f-8d4ae76eca52 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:39.742358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-773b5358-cf06-4956-ad9f-8d4ae76eca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:39.743671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:39.749828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:39.750888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:39.751878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:39.757952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 154/300] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:39.758601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:39.800764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:39.825742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 147/301] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:39.827497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae89d-c569-434d-a39d-26ca17a9d94a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.827808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae89d-c569-434d-a39d-26ca17a9d94a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.828086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae89d-c569-434d-a39d-26ca17a9d94a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.829322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 148/302] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:39.830006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ae89d-c569-434d-a39d-26ca17a9d94a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:39.874910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ffe12-265f-48b5-83cd-d24a20330de8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:39.910402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19475b8-3f1b-47dd-b4f0-c844ebda367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:39.911003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19475b8-3f1b-47dd-b4f0-c844ebda367a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.911278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19475b8-3f1b-47dd-b4f0-c844ebda367a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.911548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19475b8-3f1b-47dd-b4f0-c844ebda367a None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.912232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19475b8-3f1b-47dd-b4f0-c844ebda367a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:39.912530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19475b8-3f1b-47dd-b4f0-c844ebda367a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:39.914373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19475b8-3f1b-47dd-b4f0-c844ebda367a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:39.922868 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 155/303] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:39.924154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600fb06-4d4b-47eb-9be4-338a969e9d6b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.924434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600fb06-4d4b-47eb-9be4-338a969e9d6b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.924704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600fb06-4d4b-47eb-9be4-338a969e9d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.941517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ae89d-c569-434d-a39d-26ca17a9d94a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:39.943466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2600fb06-4d4b-47eb-9be4-338a969e9d6b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:39.944673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2600fb06-4d4b-47eb-9be4-338a969e9d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:39.968072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:39.977353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:39.978125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:39.978517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:39.978870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:39.980124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:39.981593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:39.983254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 149/304] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:39.984049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.016127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 156/305] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:40.017235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600fb06-4d4b-47eb-9be4-338a969e9d6b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.041629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:40.109116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e53cf-9bc9-43c6-b702-a7f209c58fd6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.137846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600fb06-4d4b-47eb-9be4-338a969e9d6b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.143467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.144122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.144399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.144674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] PATH_INFO: `/v3/users/e678917024d04e40b121329d26132ae5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.158957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] RBAC: Authorizing `identity:get_user(user_id=e678917024d04e40b121329d26132ae5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:40.160432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:40.162338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 150/306] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:39 2022] GET /identity/v3/users/e678917024d04e40b121329d26132ae5 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:40.163421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.180683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.181306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.181620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.181892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] PATH_INFO: `/v3/groups` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.182515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:40.183970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:40.207912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 157/307] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Sep 6 00:58:40 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:58:40.207912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.233873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:40.283144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3126d-5f00-4a1a-9fa5-40905b7cfb75 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.285432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:40.324471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.325127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.325439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.325693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.339514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:40.342652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b83d5-975c-4405-b4a9-ccdefc84b52b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:40.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:40.347258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:40.348073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 151/308] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:40.350222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.385861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.386469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.386747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.387015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.387731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] RBAC: Adding query filter params (role.id=b6eb85a18e7a483d859c5d57f92888b8, user.id=e678917024d04e40b121329d26132ae5, scope.system=all) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:40.388303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:40.390181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:40.396859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 158/309] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/role_assignments?user.id=e678917024d04e40b121329d26132ae5&scope.system=all&role.id=b6eb85a18e7a483d859c5d57f92888b8 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:40.403923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.474388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd610b5-771d-4d13-a41c-ae5eb21db6a6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.509984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:40.515721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.516098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.516382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.516655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.528486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:40.533564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:40.534500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:40.535743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 152/310] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:40.538547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e52e-0c28-4815-8369-7bd754ed6289 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.541157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.569496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:40.592552 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.593579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.593579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.593747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.594502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:40.594807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:40.596215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:40.604986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 159/311] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:40.610731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.671720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96338c12-f4b4-483a-bd47-ad485fca6f31 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.717076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.718914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.719344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.719779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.720788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:40.721574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:40.723526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:40.726876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c8c7a-9e58-4851-a1e2-78f0c68127ae None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.735178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 153/312] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:40.740685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.769317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.770773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.771345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.771831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] PATH_INFO: `/v3/users/cf033e89332647bcab0470937d465a74` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.773836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=cf033e89332647bcab0470937d465a74)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:40.776011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:40.778825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 160/313] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/users/cf033e89332647bcab0470937d465a74 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:40.786198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.807712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:40.852398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:40.876352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97c5cd-846b-4bd4-9e19-20d0d1d5eb6d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.908662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d1afb-4a30-4a15-9bbe-94a7caaa2bb7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:40.925328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.925958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.926230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.926494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] PATH_INFO: `/v3/users/1fe8f73f42c7455aba309dcab23888cd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:40.928344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] RBAC: Authorizing `identity:get_user(user_id=1fe8f73f42c7455aba309dcab23888cd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:40.929764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:40.935917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 154/314] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/users/1fe8f73f42c7455aba309dcab23888cd => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:40.936977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:40.955914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:40.956519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:40.956795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:40.957062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:41.042742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:41.045954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:41.053883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3babe33-d46c-4b9e-9931-63bc1c4b4475 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:41.064125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:41.112473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:41.113980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:41.114444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:41.114893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] PATH_INFO: `/v3/system/users/1fe8f73f42c7455aba309dcab23888cd/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:41.117076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1fe8f73f42c7455aba309dcab23888cd, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:41.119375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:41.140292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 155/315] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Sep 6 00:58:40 2022] PUT /identity/v3/system/users/1fe8f73f42c7455aba309dcab23888cd/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:41.194680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 161/316] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:41.200549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:41.221854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:41.321839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d345499-852f-4b46-a14c-371f3b28fa01 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:41.358621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb53ec-095a-497b-a45f-29137429f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:41.359261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb53ec-095a-497b-a45f-29137429f006 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:41.359539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb53ec-095a-497b-a45f-29137429f006 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:41.359830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb53ec-095a-497b-a45f-29137429f006 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:41.360962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb53ec-095a-497b-a45f-29137429f006 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:41.362405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb53ec-095a-497b-a45f-29137429f006 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:41.364491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 156/317] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:41 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:41.369905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:41.488513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77a2ee-5cbd-4ced-9e8c-1502a095d250 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:41.503840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:41.526014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2ac95-c878-404a-8f43-b9a6ead0cfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:41.526618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2ac95-c878-404a-8f43-b9a6ead0cfff None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:41.526884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2ac95-c878-404a-8f43-b9a6ead0cfff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:41.527158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2ac95-c878-404a-8f43-b9a6ead0cfff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:41.527912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2ac95-c878-404a-8f43-b9a6ead0cfff None admin] RBAC: Adding query filter params (role.id=b6eb85a18e7a483d859c5d57f92888b8, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5, user.id=cf033e89332647bcab0470937d465a74) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:41.528273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2ac95-c878-404a-8f43-b9a6ead0cfff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:41.530624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2ac95-c878-404a-8f43-b9a6ead0cfff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:41.538239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 162/318] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:58:41 2022] GET /identity/v3/role_assignments?user.id=cf033e89332647bcab0470937d465a74&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=b6eb85a18e7a483d859c5d57f92888b8 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:42.263171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7026b84-7559-44a4-8d01-c6a24a74a6b6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:42.263676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7026b84-7559-44a4-8d01-c6a24a74a6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:42.263879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7026b84-7559-44a4-8d01-c6a24a74a6b6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:42.265148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 157/319] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:42.270162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b96207-a35b-4dcf-a849-3dfc85125b9c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:42.270452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b96207-a35b-4dcf-a849-3dfc85125b9c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:42.270726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b96207-a35b-4dcf-a849-3dfc85125b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:42.289000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a5b96207-a35b-4dcf-a849-3dfc85125b9c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:42.290202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a5b96207-a35b-4dcf-a849-3dfc85125b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:42.330173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de62a-0a5d-46b9-971c-aa782a50a2d9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:42.330473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de62a-0a5d-46b9-971c-aa782a50a2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:42.330720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360de62a-0a5d-46b9-971c-aa782a50a2d9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:42.332219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 158/320] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:42.337339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc76a9-258b-4c41-b0e0-86f875d5fd36 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:42.337632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc76a9-258b-4c41-b0e0-86f875d5fd36 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:42.337905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc76a9-258b-4c41-b0e0-86f875d5fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:42.363775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f0dc76a9-258b-4c41-b0e0-86f875d5fd36 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:42.363775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f0dc76a9-258b-4c41-b0e0-86f875d5fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:42.367790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 163/321] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:42.430251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 159/322] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:42.633949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b96207-a35b-4dcf-a849-3dfc85125b9c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:42.695250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc76a9-258b-4c41-b0e0-86f875d5fd36 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:42.711126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:42.737264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b96207-a35b-4dcf-a849-3dfc85125b9c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:42.744675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:42.775590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:42.776267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:42.776554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:42.776817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:42.777923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:42.779369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:42.781587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 164/323] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:58:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:42.786905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:42.803955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc76a9-258b-4c41-b0e0-86f875d5fd36 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:42.846817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:42.847368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:42.847648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:42.847951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:42.858382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:42.863419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:42.864320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:42.865111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 160/324] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:58:42.870270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:42.893017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022748-9891-468e-a4a4-96a76ebaf426 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:42.928162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:42.928757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:42.929033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:42.929304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] PATH_INFO: `/v3/groups` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:42.929925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:42.931342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:42.956032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 165/325] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Sep 6 00:58:42 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:58:42.956864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:42.988143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cac18-9e9c-45ab-bb18-7a94d0f04454 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:43.025576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434dc52-1441-4d21-a758-cf38e8837a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:43.026197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434dc52-1441-4d21-a758-cf38e8837a13 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:43.026474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434dc52-1441-4d21-a758-cf38e8837a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:43.026743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434dc52-1441-4d21-a758-cf38e8837a13 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:43.027420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434dc52-1441-4d21-a758-cf38e8837a13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:43.027721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434dc52-1441-4d21-a758-cf38e8837a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:43.029192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434dc52-1441-4d21-a758-cf38e8837a13 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:43.039338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 161/326] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:43.045085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:43.085075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:43.146620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e1398-ea2d-4992-9fcb-f564ee48977f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:43.146912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e1398-ea2d-4992-9fcb-f564ee48977f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:43.147100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e1398-ea2d-4992-9fcb-f564ee48977f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:43.148727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 162/327] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:43.153370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305626c0-72ad-43e5-a343-2d1220727f01 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:43.153657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305626c0-72ad-43e5-a343-2d1220727f01 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:43.153932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305626c0-72ad-43e5-a343-2d1220727f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:43.158992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335d3b2-25c1-4b99-8d21-8700d4d47f5f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:43.178424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-305626c0-72ad-43e5-a343-2d1220727f01 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:43.179622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-305626c0-72ad-43e5-a343-2d1220727f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:43.196491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:43.197136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:43.197416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:43.197687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] PATH_INFO: `/v3/users/e678917024d04e40b121329d26132ae5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:43.199776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] RBAC: Authorizing `identity:get_user(user_id=e678917024d04e40b121329d26132ae5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:43.201045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:43.204042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 166/328] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:43 2022] GET /identity/v3/users/e678917024d04e40b121329d26132ae5 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:43.210731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:43.277014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 163/329] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:43.323109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2201b2-4c82-461a-ae42-0a4f2413802d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:43.324662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:43.360739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:43.361408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:43.361686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:43.361956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] PATH_INFO: `/v3/system/users/e678917024d04e40b121329d26132ae5/roles/b6eb85a18e7a483d859c5d57f92888b8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:43.363448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e678917024d04e40b121329d26132ae5, role_id=b6eb85a18e7a483d859c5d57f92888b8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:43.364865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:43.375704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 167/330] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Sep 6 00:58:43 2022] PUT /identity/v3/system/users/e678917024d04e40b121329d26132ae5/roles/b6eb85a18e7a483d859c5d57f92888b8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 00:58:43.578954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305626c0-72ad-43e5-a343-2d1220727f01 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:43.664757 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:43.706397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305626c0-72ad-43e5-a343-2d1220727f01 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:43.749258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:43.750216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:43.750498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:43.750762 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:43.763425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:43.768355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:43.769229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:43.770013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 164/331] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:43.775818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:43.900446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae940b21-57f8-4143-b8c8-448513562fb6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:43.939158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:43.939810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:43.940101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:43.940377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:43.941055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:43.941360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:43.942745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:43.955906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 168/332] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:43.957808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:44.070036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1b7cf-8502-4739-8d7e-006819f57905 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:44.099344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:44.109849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a1b56-7260-42fd-a655-5b28adbf9db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:44.110491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1b56-7260-42fd-a655-5b28adbf9db1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:44.110774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1b56-7260-42fd-a655-5b28adbf9db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:44.111112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1b56-7260-42fd-a655-5b28adbf9db1 None admin] PATH_INFO: `/v3/users/1fe8f73f42c7455aba309dcab23888cd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:44.112309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a1b56-7260-42fd-a655-5b28adbf9db1 None admin] RBAC: Authorizing `identity:get_user(user_id=1fe8f73f42c7455aba309dcab23888cd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:44.113760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a1b56-7260-42fd-a655-5b28adbf9db1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:44.119891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 165/333] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:43 2022] GET /identity/v3/users/1fe8f73f42c7455aba309dcab23888cd => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:44.121905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:44.154425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:44.232085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beaa50-27e9-42d7-9f8e-4e3050b06f18 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:44.269528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbeff8-d655-493f-a99f-e7cc65ff20e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:44.270244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbeff8-d655-493f-a99f-e7cc65ff20e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:44.270531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbeff8-d655-493f-a99f-e7cc65ff20e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:44.270805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbeff8-d655-493f-a99f-e7cc65ff20e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:44.271538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbeff8-d655-493f-a99f-e7cc65ff20e9 None admin] RBAC: Adding query filter params (role.id=cb49e0a72a2040b698c806f4871e51a5, user.id=1fe8f73f42c7455aba309dcab23888cd, scope.system=all) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:44.271875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbeff8-d655-493f-a99f-e7cc65ff20e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:44.273384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbeff8-d655-493f-a99f-e7cc65ff20e9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:44.288383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 169/334] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 6 00:58:44 2022] GET /identity/v3/role_assignments?user.id=1fe8f73f42c7455aba309dcab23888cd&scope.system=all&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:45.646335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382dcac-5ca3-4764-8b08-e4786ebd830f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:45.647247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382dcac-5ca3-4764-8b08-e4786ebd830f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:45.647669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382dcac-5ca3-4764-8b08-e4786ebd830f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:45.649483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 166/335] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:45.654484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ffedd-28f3-4a09-a02b-dd2a01b406ff None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:45.655879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ffedd-28f3-4a09-a02b-dd2a01b406ff None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:45.656301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ffedd-28f3-4a09-a02b-dd2a01b406ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:45.679507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-165ffedd-28f3-4a09-a02b-dd2a01b406ff None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:45.680959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-165ffedd-28f3-4a09-a02b-dd2a01b406ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:45.771844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 170/336] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:45.833419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c28e-422a-44ce-ae37-750e669b7c9c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:45.833916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c28e-422a-44ce-ae37-750e669b7c9c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:45.834336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c28e-422a-44ce-ae37-750e669b7c9c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:45.835851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 167/337] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:45.851857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4dd8-c25a-42e7-a833-eaf608acb7fb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:45.852395 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4dd8-c25a-42e7-a833-eaf608acb7fb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:45.852815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd4dd8-c25a-42e7-a833-eaf608acb7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:45.876360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8bcd4dd8-c25a-42e7-a833-eaf608acb7fb None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:45.877782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8bcd4dd8-c25a-42e7-a833-eaf608acb7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:45.893600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a19ce9-5906-4803-b66f-b721790c6c8b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:45.893924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a19ce9-5906-4803-b66f-b721790c6c8b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:45.894156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a19ce9-5906-4803-b66f-b721790c6c8b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:45.895689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 168/338] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:45.901597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:45.901741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399bc1a-6d31-4fcb-ba8c-6b26602cca43 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:45.901741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399bc1a-6d31-4fcb-ba8c-6b26602cca43 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:45.901741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399bc1a-6d31-4fcb-ba8c-6b26602cca43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:45.933321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3399bc1a-6d31-4fcb-ba8c-6b26602cca43 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:45.935055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3399bc1a-6d31-4fcb-ba8c-6b26602cca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:45.977363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 171/339] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:46.050490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 169/340] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:46.151700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307c03-ac09-49a8-bb9f-0f9da61d8258 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:46.152023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307c03-ac09-49a8-bb9f-0f9da61d8258 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:46.152265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307c03-ac09-49a8-bb9f-0f9da61d8258 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:46.153891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 172/341] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:46.159787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc43a6-500e-42d6-b1a7-512176d608f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:46.160090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc43a6-500e-42d6-b1a7-512176d608f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:46.160366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc43a6-500e-42d6-b1a7-512176d608f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:46.184769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5ebc43a6-500e-42d6-b1a7-512176d608f2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:46.186522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ebc43a6-500e-42d6-b1a7-512176d608f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:46.274157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 170/342] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:46.283924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307c03-ac09-49a8-bb9f-0f9da61d8258 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:46.383308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc43a6-500e-42d6-b1a7-512176d608f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:46.459929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:46.468434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307c03-ac09-49a8-bb9f-0f9da61d8258 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:46.531368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:46.532041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:46.532389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:46.532661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:46.541264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc43a6-500e-42d6-b1a7-512176d608f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:46.553066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:46.557938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:46.558795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:46.559589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 173/343] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:46.560278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:46.574186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:46.614522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:46.614522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:46.614522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:46.615042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:46.629734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:46.633363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:46.634407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 00:58:46.635189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:46.635812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 171/344] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:46.636816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:46.706760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdc5a8-3005-4d86-adb4-b29c8f996c48 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:46.749856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e7d9d-4080-4773-bd36-62a89b6f34a6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:46.754629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:46.755496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:46.755977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:46.756427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:46.768479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:46.769865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:46.775734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:46.776669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:46.777477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:46.780505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 174/345] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:46.781604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:46.799824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:46.800593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:46.800992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:46.802137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:46.802543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:46.804074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:46.816090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 172/346] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:46.817790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:46.868747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:46.916951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3779bf-ad75-4c22-ab6b-ad9fda4794c1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:46.952922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885e5ec-ee74-4b27-b46c-a3b84ca24897 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:46.956270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:46.956484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:46.956866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:46.957239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:46.969698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:46.975532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:46.976693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:46.977647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:46.979559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 175/347] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:46.980478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:46.999507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:47.000890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.001340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.001758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.002996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:47.003598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:47.008219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:47.024298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 173/348] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:47.024695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:47.091941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:47.131270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19334c-20ac-4d14-92c2-bca2bdb1edf0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:47.161324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:47.171975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:47.172643 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.172855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.173129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.173789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:47.174139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:47.175691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:47.179570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b31fc-2559-4f21-ab17-80d11a67141f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:47.184199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 176/349] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:47.185005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:47.225988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:47.226979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.227490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.228500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.302738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd506a-9cc2-4a73-a803-9b71d0c1afd3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:47.310859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:47.312736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:47.357623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:47.358632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.359113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.359582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.360500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:47.361007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:47.362021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:47.363566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:47.377520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 177/350] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:47.379261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:47.443907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 174/351] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:47.444641 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:47.453917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:47.523244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5084-f893-452d-a2c4-91dba0cc7d44 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:47.568634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbeb41-44f3-4585-b144-eeb70131a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:47.569331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbeb41-44f3-4585-b144-eeb70131a68f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.569620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbeb41-44f3-4585-b144-eeb70131a68f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.569895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbeb41-44f3-4585-b144-eeb70131a68f None admin] PATH_INFO: `/v3/users/e678917024d04e40b121329d26132ae5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.571494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbeb41-44f3-4585-b144-eeb70131a68f None admin] RBAC: Authorizing `identity:get_user(user_id=e678917024d04e40b121329d26132ae5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:47.573121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbeb41-44f3-4585-b144-eeb70131a68f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:47.574881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 178/352] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 00:58:47 2022] GET /identity/v3/users/e678917024d04e40b121329d26132ae5 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:47.576054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba8f1-7664-4a4e-a5dd-6179d283b17b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.576338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba8f1-7664-4a4e-a5dd-6179d283b17b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.577138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba8f1-7664-4a4e-a5dd-6179d283b17b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.583295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 179/353] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:47.583473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ba8f1-7664-4a4e-a5dd-6179d283b17b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:47.590523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a1fca-a2c9-4caf-a16a-992457eeda4f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:47.629810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:47.630429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.630708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.630981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.641906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:47.721172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ba8f1-7664-4a4e-a5dd-6179d283b17b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:47.722929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:47.726745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:47.746870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:47.776864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:47.777835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:47.778122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:47.778396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:47.859980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:47.861369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:47.867176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 175/354] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:47.868061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:47.921101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:47.987144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 180/355] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:47.989732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23d97b-755b-4237-bb9b-b614b977ccd5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:47.992733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.013740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:48.030832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39129bf1-83a1-4cfc-8072-06723b207e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.032416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39129bf1-83a1-4cfc-8072-06723b207e42 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.032710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39129bf1-83a1-4cfc-8072-06723b207e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.033048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39129bf1-83a1-4cfc-8072-06723b207e42 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.035697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39129bf1-83a1-4cfc-8072-06723b207e42 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.037612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39129bf1-83a1-4cfc-8072-06723b207e42 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.040178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 176/356] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:47 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.041657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f7890-5594-494e-97b1-fd23089a87a9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.042061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f7890-5594-494e-97b1-fd23089a87a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.042344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f7890-5594-494e-97b1-fd23089a87a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.063415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1f3f7890-5594-494e-97b1-fd23089a87a9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:48.064688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f3f7890-5594-494e-97b1-fd23089a87a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:48.117558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e2330-cfea-4ce3-9b73-bd89b2efedd8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.145129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 177/357] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:48.145771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f7890-5594-494e-97b1-fd23089a87a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.159652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca7968-4777-4df3-b9a8-f8758ecb3039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.159652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca7968-4777-4df3-b9a8-f8758ecb3039 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.159964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca7968-4777-4df3-b9a8-f8758ecb3039 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.160248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca7968-4777-4df3-b9a8-f8758ecb3039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.160875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca7968-4777-4df3-b9a8-f8758ecb3039 None admin] RBAC: Adding query filter params (role.id=b6eb85a18e7a483d859c5d57f92888b8, user.id=e678917024d04e40b121329d26132ae5, scope.system=all) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:48.161354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca7968-4777-4df3-b9a8-f8758ecb3039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.162620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca7968-4777-4df3-b9a8-f8758ecb3039 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.177081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 181/358] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 6 00:58:47 2022] GET /identity/v3/role_assignments?user.id=e678917024d04e40b121329d26132ae5&scope.system=all&role.id=b6eb85a18e7a483d859c5d57f92888b8 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.178145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90731-124a-40cb-ba97-2db69d3869b1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.178430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90731-124a-40cb-ba97-2db69d3869b1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.178705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b90731-124a-40cb-ba97-2db69d3869b1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.180117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 182/359] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:48.180681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90731-124a-40cb-ba97-2db69d3869b1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.190105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:48.283608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f7890-5594-494e-97b1-fd23089a87a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.287665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:48.330324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b90731-124a-40cb-ba97-2db69d3869b1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.346598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.347617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.348246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.348784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.350188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.351922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.353850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 178/360] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.354603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.375824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d8a87-dc40-4692-b2a0-9df07b2be488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.376510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d8a87-dc40-4692-b2a0-9df07b2be488 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.376794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d8a87-dc40-4692-b2a0-9df07b2be488 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.377072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d8a87-dc40-4692-b2a0-9df07b2be488 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.378252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d8a87-dc40-4692-b2a0-9df07b2be488 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.379844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d8a87-dc40-4692-b2a0-9df07b2be488 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.381572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 183/361] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.382760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de198b75-518a-4a73-8c67-c94a0f9cc6ed None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.383049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de198b75-518a-4a73-8c67-c94a0f9cc6ed None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.383324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de198b75-518a-4a73-8c67-c94a0f9cc6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.408912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-de198b75-518a-4a73-8c67-c94a0f9cc6ed None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:48.410121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-de198b75-518a-4a73-8c67-c94a0f9cc6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:48.485517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548ae02-5653-4e83-89c7-10958facd309 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.489109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:48.489803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 184/362] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:48.490938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de198b75-518a-4a73-8c67-c94a0f9cc6ed None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.531224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.531889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.532167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.532440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.539414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.540924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.542736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 179/363] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.545083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.587745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:48.622125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de198b75-518a-4a73-8c67-c94a0f9cc6ed None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.663620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.664222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.664502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.664772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.665884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.668257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.670380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 185/364] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.670380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.684464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35cdf8-a5e3-4ca8-8c3e-7df20a0e82ea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.736514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.737194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.737475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.737750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.738990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.740459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.742220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 180/365] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.743265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.799130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:48.804062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ed66-f1e2-4649-a8eb-e74958c3379c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.848087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a565b-1201-41cd-b331-282504831da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.849128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a565b-1201-41cd-b331-282504831da1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.849128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a565b-1201-41cd-b331-282504831da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.849320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a565b-1201-41cd-b331-282504831da1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.864096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a565b-1201-41cd-b331-282504831da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-385a565b-1201-41cd-b331-282504831da1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:48.869282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:48.870174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:48.870977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 186/366] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:48.871647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a565b-1201-41cd-b331-282504831da1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:48.876869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:48.894199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a85fae-7c7f-4c60-a667-0de1625442d8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:48.937807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:48.939360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:48.939360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:48.939360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:48.939740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:48.940071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:48.941492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:48.947778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 181/367] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:48.948447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:49.015389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a565b-1201-41cd-b331-282504831da1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:49.070820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:49.071795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:49.071795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:49.072090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:49.072837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=66704401130047e78e714312f45defd9, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:49.073147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:49.074628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:49.084013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 187/368] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=66704401130047e78e714312f45defd9 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:49.084013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:49.085648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:49.089562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d41e5b-b480-4af0-8825-e0e2a7c00beb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:49.151789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:49.151789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:49.151789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:49.151789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:49.152212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:49.152212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:49.157387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:49.164719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 182/369] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:48 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:49.166038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:49.206864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:49.312265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0387-0fa9-4f20-9870-401478a26bb8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:49.318717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9644e-4c1e-48d4-a280-7c2c27021a59 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:49.355847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:49.356776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:49.357521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:49.358662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:49.359560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:49.360058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:49.362165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:49.372524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 183/370] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:49.378764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:49.397070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:49.399136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:49.399587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:49.400047 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:49.412436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:49.418644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:49.419537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:49.420945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:49.423142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 188/371] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:49.428865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:49.442382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:49.590964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3c11-098e-4423-8d3d-f8dee68f0bf4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:49.637373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:49.637980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:49.638257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:49.638525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:49.673718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda46d2d-d965-4520-8bfe-3bfaff2db3ef None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:49.720002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:49.722004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:49.744221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:49.756066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:49.784471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:49.785102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:49.785382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:49.785655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:49.786329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:49.786630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:49.788054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:49.803226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 189/372] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:49.812756 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:49.834980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 184/373] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:49.840741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:49.946155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c84ef1-58f9-41c7-8855-97e88403f3df None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:49.974582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:49.984484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:49.985129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:49.985413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:49.985692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:49.986849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:49.988334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:49.994546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 185/374] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:49 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:49.995763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:50.004953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592826b-cea3-4907-92d4-f4768c040ca3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:50.080273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:50.080882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:50.081162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:50.081431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:50.103259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001be63b-a005-4a69-af9a-25971434b4c7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:50.138979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:50.140208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:50.140489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:50.140765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:50.142709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:50.144185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:50.150630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 186/375] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:49 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:50.151879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:50.192105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:50.214461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:50.216006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:50.275205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:50.344333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d8554-d0ed-4c27-b785-287434dacd51 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:50.409096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 190/376] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:50.411648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:50.412291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:50.412571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:50.412842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:50.413819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=66704401130047e78e714312f45defd9, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:50.414124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:50.415551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:50.417354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:50.430069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 187/377] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:50 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=66704401130047e78e714312f45defd9 => generated 260 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:50.550971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fcc03-ebba-47cf-b4f1-ef469ff6f164 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:50.552692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:50.588618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:50.589443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:50.589897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:50.590303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] PATH_INFO: `/v3/projects/921fe46fc085463aa0816bc6ba906e3c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:50.591616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] RBAC: Authorizing `identity:get_project(project_id=921fe46fc085463aa0816bc6ba906e3c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:50.593370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:50.595567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 191/378] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:50 2022] GET /identity/v3/projects/921fe46fc085463aa0816bc6ba906e3c => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:50.604925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:50.743266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f85af1-636c-4049-babd-4bbadba7eb1f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:50.777149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:50.790316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1da10-2259-4320-ad80-f3f548ece74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:50.790943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1da10-2259-4320-ad80-f3f548ece74b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:50.791220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1da10-2259-4320-ad80-f3f548ece74b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:50.791489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1da10-2259-4320-ad80-f3f548ece74b None admin] PATH_INFO: `/v3/groups/e9c9fbf316ac48fc9314c37c27e3f1ba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:50.798706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1da10-2259-4320-ad80-f3f548ece74b None admin] RBAC: Authorizing `identity:get_group(group_id=e9c9fbf316ac48fc9314c37c27e3f1ba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:50.800183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1da10-2259-4320-ad80-f3f548ece74b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:50.802050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 188/379] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:50 2022] GET /identity/v3/groups/e9c9fbf316ac48fc9314c37c27e3f1ba => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:50.808097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:50.933979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9aadc-b229-4a6c-934e-e96ff4c3b19e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:50.978926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ce5df-9a82-4977-81e6-b36ddd5dba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:50.979906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ce5df-9a82-4977-81e6-b36ddd5dba56 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:50.980405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ce5df-9a82-4977-81e6-b36ddd5dba56 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:50.980883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ce5df-9a82-4977-81e6-b36ddd5dba56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:50.981803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ce5df-9a82-4977-81e6-b36ddd5dba56 None admin] RBAC: Adding query filter params (group.id=e9c9fbf316ac48fc9314c37c27e3f1ba, role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=921fe46fc085463aa0816bc6ba906e3c) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:50.982301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ce5df-9a82-4977-81e6-b36ddd5dba56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:50.984505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ce5df-9a82-4977-81e6-b36ddd5dba56 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:50.990416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 192/380] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:50 2022] GET /identity/v3/role_assignments?group.id=e9c9fbf316ac48fc9314c37c27e3f1ba&scope.project.id=921fe46fc085463aa0816bc6ba906e3c&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:51.444758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b544bc-89d6-47cc-9638-04def575f517 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:51.445267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b544bc-89d6-47cc-9638-04def575f517 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:51.445339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b544bc-89d6-47cc-9638-04def575f517 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:51.446913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 189/381] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:51.452236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bac68-7a6d-4026-9fac-c4a01832c845 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:51.452730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bac68-7a6d-4026-9fac-c4a01832c845 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:51.453210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bac68-7a6d-4026-9fac-c4a01832c845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:51.476646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fe8bac68-7a6d-4026-9fac-c4a01832c845 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:51.478104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fe8bac68-7a6d-4026-9fac-c4a01832c845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:51.497413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:51.517585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e206c-e1d8-4617-9b17-6bc49ddeff3c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:51.517891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e206c-e1d8-4617-9b17-6bc49ddeff3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:51.518128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e206c-e1d8-4617-9b17-6bc49ddeff3c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:51.523776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 190/382] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:58:51.524581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f109fa-079b-49a8-88ba-b0047411aad8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:51.524866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f109fa-079b-49a8-88ba-b0047411aad8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:51.525144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f109fa-079b-49a8-88ba-b0047411aad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:51.543461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-19f109fa-079b-49a8-88ba-b0047411aad8 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:51.544842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-19f109fa-079b-49a8-88ba-b0047411aad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:51.579018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 193/383] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:51.657611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 191/384] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:51.722806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ffd4d-bc67-410a-a54c-441918d82d90 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:51.723154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ffd4d-bc67-410a-a54c-441918d82d90 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:51.723522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ffd4d-bc67-410a-a54c-441918d82d90 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:51.725008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 194/385] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:51.741042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8cc4-0a9c-4ee5-990e-d6c00113f3e7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:51.741622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8cc4-0a9c-4ee5-990e-d6c00113f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:51.742093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d8cc4-0a9c-4ee5-990e-d6c00113f3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:51.763732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fc6d8cc4-0a9c-4ee5-990e-d6c00113f3e7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:51.765812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc6d8cc4-0a9c-4ee5-990e-d6c00113f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:51.853376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 192/386] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:51.892568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ffd4d-bc67-410a-a54c-441918d82d90 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.022351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8cc4-0a9c-4ee5-990e-d6c00113f3e7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.029693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ffd4d-bc67-410a-a54c-441918d82d90 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.098608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.099241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.099517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.099801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.120672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:52.126402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:52.127278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:52.128085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:52.130224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.130895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 195/387] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:52.139242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.169295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.190497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d8cc4-0a9c-4ee5-990e-d6c00113f3e7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.226019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.226991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.227472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.227993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.239852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:52.246553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:52.247546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:52.248402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:52.250773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 193/388] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:58:52.256705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.320885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a59dd-bc8b-4d4c-abe5-cba3799faa0f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.356866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.357957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.358983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.359445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.360350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:52.360847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:52.362455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:52.370788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e9da2-60bd-471c-81b2-0c9273574ca6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.376205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 196/389] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:52.377324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.382621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.411203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.411914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.412196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.412473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.413139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:52.413694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:52.415075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:52.424773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 194/390] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:52.425603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.455551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.489226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dd7c7-4277-49ad-9434-49bab00dd5e6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.525918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.527011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.527730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.528641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.531816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8368cfa-82f0-44b0-8d62-88550062cbb1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.543874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:52.551311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:52.552195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:52.552998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:52.556344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 197/391] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:52.558082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.575465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.576473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.577469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.577937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.579286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:52.580969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:52.595786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 195/392] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:52.595786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.631595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.668734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd5d1-8a22-4807-9a03-5fc644f692f5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.694401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee201bf-57fc-4658-b89d-3071b2aa88c6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.698882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.705762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.706403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.707034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.707322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.708511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:52.709965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:52.711886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 198/393] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:52.712723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.734787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.735720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.736212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.736711 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.737548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:52.738022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:52.739719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:52.749359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 196/394] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:52.750615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.824087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9a51-90ad-451b-a013-2a10990ec11e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.862615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.863266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.863571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.863882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.865004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:52.866460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:52.872709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 199/395] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:52.872868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c88ff9-6441-4644-89d8-e5ecb9829ff0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:52.872977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.873857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.910899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93938ad-306b-4c7a-9337-b2db060b8cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:52.911525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93938ad-306b-4c7a-9337-b2db060b8cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.911824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93938ad-306b-4c7a-9337-b2db060b8cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.912092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93938ad-306b-4c7a-9337-b2db060b8cf0 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.913210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93938ad-306b-4c7a-9337-b2db060b8cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:52.914902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93938ad-306b-4c7a-9337-b2db060b8cf0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:52.916549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 197/396] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:52.921736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f07b8-9933-4faa-a348-a65c989fc4a1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:52.922022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f07b8-9933-4faa-a348-a65c989fc4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:52.922293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f07b8-9933-4faa-a348-a65c989fc4a1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:52.923786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 198/397] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:52.924236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f07b8-9933-4faa-a348-a65c989fc4a1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:52.943025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:52.981455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91516-5719-4e28-92b4-6f44fd6672ff None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:53.017930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:53.018843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.019137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.019414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.020716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:53.022872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:53.025092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 200/398] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:52 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:53.026050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:53.037347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f07b8-9933-4faa-a348-a65c989fc4a1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:53.072172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490836-6fff-4ea8-9d69-60239bfa559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:53.072831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490836-6fff-4ea8-9d69-60239bfa559c None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.073104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490836-6fff-4ea8-9d69-60239bfa559c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.073372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490836-6fff-4ea8-9d69-60239bfa559c None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/groups/3c1adf6daaa04c7c9152a98692962a42/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.075224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490836-6fff-4ea8-9d69-60239bfa559c None admin] RBAC: Authorizing `identity:create_grant(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd, group_id=3c1adf6daaa04c7c9152a98692962a42, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:53.077212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490836-6fff-4ea8-9d69-60239bfa559c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:53.099793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 199/399] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 6 00:58:52 2022] PUT /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/groups/3c1adf6daaa04c7c9152a98692962a42/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:53.099793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a113-e91f-41e8-9abe-cd2ac1deb5ca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.099793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a113-e91f-41e8-9abe-cd2ac1deb5ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.099793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a113-e91f-41e8-9abe-cd2ac1deb5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.121299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a608a113-e91f-41e8-9abe-cd2ac1deb5ca None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:53.122929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a608a113-e91f-41e8-9abe-cd2ac1deb5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:53.153778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2828149-af72-4963-b3e1-dc0ac3f0bc1e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:53.158827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:53.205875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635d38d-196c-41fc-bf54-45b7329be982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:53.206829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d38d-196c-41fc-bf54-45b7329be982 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.207196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d38d-196c-41fc-bf54-45b7329be982 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.207901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d38d-196c-41fc-bf54-45b7329be982 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/groups/3c1adf6daaa04c7c9152a98692962a42/roles/66704401130047e78e714312f45defd9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.211067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635d38d-196c-41fc-bf54-45b7329be982 None admin] RBAC: Authorizing `identity:create_grant(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd, group_id=3c1adf6daaa04c7c9152a98692962a42, role_id=66704401130047e78e714312f45defd9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:53.212974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635d38d-196c-41fc-bf54-45b7329be982 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:53.215591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 200/400] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:53.216997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a113-e91f-41e8-9abe-cd2ac1deb5ca None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:53.219314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:53.232567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 201/401] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 6 00:58:53 2022] PUT /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd/groups/3c1adf6daaa04c7c9152a98692962a42/roles/66704401130047e78e714312f45defd9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:53.304322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa57eb1-a942-4254-830c-a89da2af63f1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.304675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa57eb1-a942-4254-830c-a89da2af63f1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.304872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa57eb1-a942-4254-830c-a89da2af63f1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.306517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 202/402] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:53.312849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9d84-fccf-4964-9aa5-e7f7120cc805 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.313179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9d84-fccf-4964-9aa5-e7f7120cc805 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.313396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9d84-fccf-4964-9aa5-e7f7120cc805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.334612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a2bd9d84-fccf-4964-9aa5-e7f7120cc805 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:53.336525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a2bd9d84-fccf-4964-9aa5-e7f7120cc805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:53.345590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a113-e91f-41e8-9abe-cd2ac1deb5ca None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:53.391694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:53.392292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.392570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.392841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.394736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:53.396240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:53.398861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 201/403] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:53 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:53.404982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:53.433307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 203/404] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:58:53.444014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:53.520823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f01c24-925e-4e66-bbf2-5f3112714eea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:53.528835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9d84-fccf-4964-9aa5-e7f7120cc805 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:53.583959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:53.585167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.586915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.587402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/groups/3c1adf6daaa04c7c9152a98692962a42/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.590733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] RBAC: Authorizing `identity:create_grant(project_id=fb680eb41a4a40bcb931685c4b20c1b5, group_id=3c1adf6daaa04c7c9152a98692962a42, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:53.593776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:53.619423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 202/405] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 6 00:58:53 2022] PUT /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/groups/3c1adf6daaa04c7c9152a98692962a42/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 00:58:53.703215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9d84-fccf-4964-9aa5-e7f7120cc805 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:53.726275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:53.750049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:53.768531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:53.769242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.769524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.769794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.786489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:53.792532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:53.793499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:53.794593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 204/406] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:53.802180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:53.856726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:53.870885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506dcd3-85f7-42fc-9d36-5b699e56877e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:53.905816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:53.906448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:53.906721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:53.906986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:53.917709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:53.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:53.924278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:53.925685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 203/407] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:58:53.929806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:53.942725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215803a-81a4-4167-b4ba-6caf96927697 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.005741 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.006474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.006754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.007023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.007691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:54.008014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.009384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.024512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 205/408] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:54.031766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:54.095328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce992629-892f-4d49-b6c4-077b74ac8344 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.118726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:54.160497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.161103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.161380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.161650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.162305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:54.162607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.164312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.179298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 204/409] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:54.184361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:54.203057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:54.212946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16820e8e-57b9-494e-8e17-394628f58104 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.278555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.279105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.280367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.280709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.281976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.283425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.285008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 206/410] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:54 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:54.292325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:54.372984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ea99f-ea04-4b2c-991d-481b299d4fa7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.441450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.442450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.442949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.443436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] PATH_INFO: `/v3/groups/e9c9fbf316ac48fc9314c37c27e3f1ba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.446132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] RBAC: Authorizing `identity:get_group(group_id=e9c9fbf316ac48fc9314c37c27e3f1ba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.447811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.452479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 205/411] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:54 2022] GET /identity/v3/groups/e9c9fbf316ac48fc9314c37c27e3f1ba => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:54.460685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:54.470353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b59e1-2695-466c-b40d-dbc4491937a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.489004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:54.512493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.513475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.513975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.514984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.516716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.518382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.520945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 207/412] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:54 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:54.528623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:54.645874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:54.681918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec64b1-1e04-4045-bb96-f6c3ecb36b5b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.685102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee7a1-2a9a-4661-9604-7de6102425d5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.741812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.742451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.742732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.743008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] PATH_INFO: `/v3/projects/921fe46fc085463aa0816bc6ba906e3c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.745003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa64640-cf01-41d3-966e-3d6ad38a5411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.745674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64640-cf01-41d3-966e-3d6ad38a5411 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.745950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64640-cf01-41d3-966e-3d6ad38a5411 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.746219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa64640-cf01-41d3-966e-3d6ad38a5411 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/groups/3c1adf6daaa04c7c9152a98692962a42/roles/66704401130047e78e714312f45defd9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.747440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] RBAC: Authorizing `identity:get_project(project_id=921fe46fc085463aa0816bc6ba906e3c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.749570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.753442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa64640-cf01-41d3-966e-3d6ad38a5411 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb680eb41a4a40bcb931685c4b20c1b5, group_id=3c1adf6daaa04c7c9152a98692962a42, role_id=66704401130047e78e714312f45defd9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.754936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa64640-cf01-41d3-966e-3d6ad38a5411 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.757605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 206/413] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:54 2022] GET /identity/v3/projects/921fe46fc085463aa0816bc6ba906e3c => generated 344 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:54.758547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:54.779451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 208/414] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 6 00:58:54 2022] PUT /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5/groups/3c1adf6daaa04c7c9152a98692962a42/roles/66704401130047e78e714312f45defd9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:54.920043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8cd0-0f02-4dac-adb2-00c85197014e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:54.966922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78992e58-26f4-4e1d-890b-d3bc8a1e2b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:54.967620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992e58-26f4-4e1d-890b-d3bc8a1e2b9e None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:54.967928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992e58-26f4-4e1d-890b-d3bc8a1e2b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:54.968202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78992e58-26f4-4e1d-890b-d3bc8a1e2b9e None admin] PATH_INFO: `/v3/projects/921fe46fc085463aa0816bc6ba906e3c/groups/e9c9fbf316ac48fc9314c37c27e3f1ba/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:54.970080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992e58-26f4-4e1d-890b-d3bc8a1e2b9e None admin] RBAC: Authorizing `identity:create_grant(project_id=921fe46fc085463aa0816bc6ba906e3c, group_id=e9c9fbf316ac48fc9314c37c27e3f1ba, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:54.971555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78992e58-26f4-4e1d-890b-d3bc8a1e2b9e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:54.993299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 207/415] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 6 00:58:54 2022] PUT /identity/v3/projects/921fe46fc085463aa0816bc6ba906e3c/groups/e9c9fbf316ac48fc9314c37c27e3f1ba/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:58:55.483949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b2051-d5ed-46a3-8bc9-e6d44470e35f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:55.484454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b2051-d5ed-46a3-8bc9-e6d44470e35f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:55.484528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b2051-d5ed-46a3-8bc9-e6d44470e35f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:55.486210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 209/416] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:55.494883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd954-f153-4d32-9409-513d4d3279f4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:55.494883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd954-f153-4d32-9409-513d4d3279f4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:55.494883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd954-f153-4d32-9409-513d4d3279f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:55.506278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:55.512177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b24bd954-f153-4d32-9409-513d4d3279f4 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:55.513685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b24bd954-f153-4d32-9409-513d4d3279f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:55.626212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 208/417] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:55.684535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09a43d-5876-4ae8-adfe-1a28847e6fbd None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:55.684835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09a43d-5876-4ae8-adfe-1a28847e6fbd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:55.685089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09a43d-5876-4ae8-adfe-1a28847e6fbd None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:55.686717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 210/418] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:55.691856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3eb9-101c-453c-9612-569afd1e6e54 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:55.692673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3eb9-101c-453c-9612-569afd1e6e54 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:55.693148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3eb9-101c-453c-9612-569afd1e6e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:55.715476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e8ce3eb9-101c-453c-9612-569afd1e6e54 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:55.717069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e8ce3eb9-101c-453c-9612-569afd1e6e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:55.805465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 209/419] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:55.943129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbe9c-ab12-4cec-b87f-c9bd6c8c6884 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:55.943446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbe9c-ab12-4cec-b87f-c9bd6c8c6884 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:55.943691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbe9c-ab12-4cec-b87f-c9bd6c8c6884 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:55.945173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 211/420] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:55.950356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d3b6-2e38-4597-9064-f1d768849a17 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:55.950648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d3b6-2e38-4597-9064-f1d768849a17 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:55.950920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d3b6-2e38-4597-9064-f1d768849a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:55.980737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2401d3b6-2e38-4597-9064-f1d768849a17 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:55.981913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2401d3b6-2e38-4597-9064-f1d768849a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:56.039204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbbe9c-ab12-4cec-b87f-c9bd6c8c6884 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.058724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:56.071833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 210/421] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:56.121974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:56.198357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbbe9c-ab12-4cec-b87f-c9bd6c8c6884 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:56.231584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401d3b6-2e38-4597-9064-f1d768849a17 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.238987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:56.239627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:56.239933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:56.240208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:56.251373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:56.257134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:56.257981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:58:56.258751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 212/422] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:56.265927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.392889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401d3b6-2e38-4597-9064-f1d768849a17 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:56.422920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3093877-8453-46f0-8c5b-c0039f609dcf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:56.425821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:56.432317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:56.433342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:56.433627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:56.433901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:56.447460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:56.454423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:56.455349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:56.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 211/423] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:58:56.456818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.468314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:56.469807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:56.470654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:56.471619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:56.472917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:56.473568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:56.475253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:56.490591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 213/424] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:56.492336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.540849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:56.594034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726b2a-112b-4707-af99-0dbc71bf6c82 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:56.619561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a074d-2809-4226-8903-00f747b47ad7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:56.632629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:56.633565 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:56.634140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:56.634606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:56.646251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:56.651716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:56.652627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 00:58:56.653425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:56.660938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:56.660938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:56.660938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:56.660938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:56.661453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 212/425] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:56.661566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:56.661775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:56.662748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.663150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:56.675897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 214/426] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:56.677084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.715162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:56.777562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6a6fe-5575-41cc-b1cf-6dd19a14a217 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:56.797316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:56.809768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a4c3f-5d52-4659-aeee-f515128a34e8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:56.818987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:56.819903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:56.820383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:56.820963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:56.851521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:56.852189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:56.852859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:56.853140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:56.854129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:56.854440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:56.856085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:56.872568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 215/427] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:56.872568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:56.911645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:56.913094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:56.972909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:56.986607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fb203-02a1-47fd-a432-d75c8eb089d5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.028526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3eeb-acea-43a4-9c3c-58cfb37ac210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.029138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3eeb-acea-43a4-9c3c-58cfb37ac210 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.029416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3eeb-acea-43a4-9c3c-58cfb37ac210 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.029710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3eeb-acea-43a4-9c3c-58cfb37ac210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.030695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 213/428] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:57.031855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.052367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.105547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3eeb-acea-43a4-9c3c-58cfb37ac210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.107012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3eeb-acea-43a4-9c3c-58cfb37ac210 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.140540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553df71-a9e0-4933-bbff-5b9da27bb46c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.181138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce08aa-7511-4237-b94c-683b71ef95a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.181753 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce08aa-7511-4237-b94c-683b71ef95a8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.182033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce08aa-7511-4237-b94c-683b71ef95a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.182309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce08aa-7511-4237-b94c-683b71ef95a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.198698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.211138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 216/429] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:57.212481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f357c1f-3726-4e84-879a-c99e266cda3b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.212834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f357c1f-3726-4e84-879a-c99e266cda3b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.213177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f357c1f-3726-4e84-879a-c99e266cda3b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.219159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 217/430] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:57.219159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f357c1f-3726-4e84-879a-c99e266cda3b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.263679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce08aa-7511-4237-b94c-683b71ef95a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.266200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce08aa-7511-4237-b94c-683b71ef95a8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.284363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.316784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f357c1f-3726-4e84-879a-c99e266cda3b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.350360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.350986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.351262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.351535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.352691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.354126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.355731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 218/431] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:57.356499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.379846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 214/432] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:58:57.381413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66583330-c014-43ad-ae4c-771dd892a6aa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.381705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66583330-c014-43ad-ae4c-771dd892a6aa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.381979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66583330-c014-43ad-ae4c-771dd892a6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.400098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-66583330-c014-43ad-ae4c-771dd892a6aa None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:57.401290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-66583330-c014-43ad-ae4c-771dd892a6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:57.421144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.464326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6863d1-ee5f-40d6-bf8a-2172e5b3e757 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.467645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 215/433] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:57.468515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66583330-c014-43ad-ae4c-771dd892a6aa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.499532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.500236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.500978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.501257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] PATH_INFO: `/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.502598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] RBAC: Authorizing `identity:get_project(project_id=da9a2e0ae36a4eb58ba0915ef0f274bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.504869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.507541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 219/434] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/projects/da9a2e0ae36a4eb58ba0915ef0f274bd => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:57.508141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.528363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.592006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66583330-c014-43ad-ae4c-771dd892a6aa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.618859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050758ac-2532-4267-9398-6779edcb2360 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.629336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37482e57-62d3-4c1f-bd2f-e093d89c00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.630273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482e57-62d3-4c1f-bd2f-e093d89c00ae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.630564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482e57-62d3-4c1f-bd2f-e093d89c00ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.630990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37482e57-62d3-4c1f-bd2f-e093d89c00ae None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.632144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37482e57-62d3-4c1f-bd2f-e093d89c00ae None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.633560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37482e57-62d3-4c1f-bd2f-e093d89c00ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.635115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 216/435] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:57.636231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554fb2-b252-4781-9d00-72001bb42fd3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.636510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554fb2-b252-4781-9d00-72001bb42fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.636780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64554fb2-b252-4781-9d00-72001bb42fd3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.642414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 217/436] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:58:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:58:57.642414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554fb2-b252-4781-9d00-72001bb42fd3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.652475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.662156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.662850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.663178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.663454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.664962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.666400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.668131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 220/437] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:57.668131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.751835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64554fb2-b252-4781-9d00-72001bb42fd3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.766727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.772839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e5b33-9be2-46d4-bbbd-f6e0789cae39 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.786758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94381bb3-40ca-488f-9158-e0ea229f0204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.787397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381bb3-40ca-488f-9158-e0ea229f0204 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.787676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381bb3-40ca-488f-9158-e0ea229f0204 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.787979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381bb3-40ca-488f-9158-e0ea229f0204 None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.789380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94381bb3-40ca-488f-9158-e0ea229f0204 None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.790887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94381bb3-40ca-488f-9158-e0ea229f0204 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.792632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 218/438] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:57.793869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd5f53-ca0a-467d-af8b-c8791c94faf1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.794160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd5f53-ca0a-467d-af8b-c8791c94faf1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.794436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd5f53-ca0a-467d-af8b-c8791c94faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.808772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.809373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.809649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.809919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.810636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:57.810963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.812853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9ffd5f53-ca0a-467d-af8b-c8791c94faf1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:58:57.812969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.814017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9ffd5f53-ca0a-467d-af8b-c8791c94faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:58:57.818639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 221/439] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:57.819515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.888691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 219/440] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:58:57.889455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd5f53-ca0a-467d-af8b-c8791c94faf1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:57.904642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:57.954258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036848-275f-44bd-ae33-5d802436a5e1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:57.991497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:57.992172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:57.992445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:57.992747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:57.993845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:57.995342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:57.996923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 222/441] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:57.997676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.024407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd5f53-ca0a-467d-af8b-c8791c94faf1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.040202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:58.072866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.072866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.073158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.073240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.087200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:58.093186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:58.094108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:58.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 220/442] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:58.095523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.140945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a057e3-7957-4b87-b842-e2a3c7d05e4e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.173774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:58.184242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.184860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.185134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.185405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.186163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=66704401130047e78e714312f45defd9, scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:58.186469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:58.187993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:58.197176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 223/443] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:57 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=da9a2e0ae36a4eb58ba0915ef0f274bd&role.id=66704401130047e78e714312f45defd9 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:58.198005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.222662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ccd72-6539-4e65-98ff-f8353af4b643 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.284384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.285324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.285922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.286905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.289046 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=cb49e0a72a2040b698c806f4871e51a5, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:58.289364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:58.290962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:58.298303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 221/444] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=cb49e0a72a2040b698c806f4871e51a5 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:58.299281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.347037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee2ed-27fc-4a0d-9c48-6910ae9b6afa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.355016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:58.403442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.404103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.404384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.404658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.405423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:58.405899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:58.407320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:58.418131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 224/445] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:58.423709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.457784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce004fe-ea7d-4aa3-a63c-5d5f1f32b22f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.504277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:58.513049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.513684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.514020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.514852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.530936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:58:58.536156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:58:58.537248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:58:58.538389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 222/446] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:58:58.543449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.575061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180665d-2cac-4e3d-a4a3-c113a51c6557 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.621557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.622171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.622451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.622721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.660779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:58.668742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e62eca-484a-4a88-aa24-38691749a705 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.710461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:58.711664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.712334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.712529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:58.712652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.712880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.713545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:58.713848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:58.715210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:58.724828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 223/447] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:58.730336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.777391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:58.847223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 225/448] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 00:58:58.852760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:58.857190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0f6a2-8546-4794-87e1-aa24dd47aa23 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.890429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.891021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.891302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.891573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.917680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:58.947958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f08db-f975-4711-8f89-ca5af3bf857e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:58.957957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:58.959401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:58.981954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:58.982581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:58.982887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:58.983161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] PATH_INFO: `/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:58.984287 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] RBAC: Authorizing `identity:get_project(project_id=fb680eb41a4a40bcb931685c4b20c1b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:58.985715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:58.987308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 226/449] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/projects/fb680eb41a4a40bcb931685c4b20c1b5 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:58.992693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:59.013509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:59.055651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 224/450] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 00:58:59.062557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:59.089405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa263a-6986-4c4c-a4b5-6090576331c7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:59.116604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:59.122102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:59.122725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:59.123007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:59.123275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] PATH_INFO: `/v3/groups/3c1adf6daaa04c7c9152a98692962a42` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:59.125064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] RBAC: Authorizing `identity:get_group(group_id=3c1adf6daaa04c7c9152a98692962a42)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:59.126502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:59.132967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 227/451] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:58 2022] GET /identity/v3/groups/3c1adf6daaa04c7c9152a98692962a42 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:59.133716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:59.158130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e77180-f30d-44f4-8c77-c2da40d7e3b2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:59.191300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:59.191950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:59.192227 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:59.192498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] PATH_INFO: `/v3/projects/921fe46fc085463aa0816bc6ba906e3c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:59.193681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=921fe46fc085463aa0816bc6ba906e3c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:59.195118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:59.196825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 225/452] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 00:58:59 2022] GET /identity/v3/projects/921fe46fc085463aa0816bc6ba906e3c => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:58:59.202911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:59.219337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:59.231423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691bd6c-97bd-4a13-a5e8-7612103a8b24 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:59.264740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:59.265339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:59.265620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:59.265890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:59.266601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] RBAC: Adding query filter params (group.id=3c1adf6daaa04c7c9152a98692962a42, role.id=66704401130047e78e714312f45defd9, scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:59.266902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:59.268351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:59.273898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 228/453] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:59 2022] GET /identity/v3/role_assignments?group.id=3c1adf6daaa04c7c9152a98692962a42&scope.project.id=fb680eb41a4a40bcb931685c4b20c1b5&role.id=66704401130047e78e714312f45defd9 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:59.301222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75b4e1-ac88-4b68-a4d1-6a97b01c8ca0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:59.328880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:58:59.337965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3abf2-8c2b-4795-9af0-fb3db85bf711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:59.338606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3abf2-8c2b-4795-9af0-fb3db85bf711 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:59.338883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3abf2-8c2b-4795-9af0-fb3db85bf711 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:59.339157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3abf2-8c2b-4795-9af0-fb3db85bf711 None admin] PATH_INFO: `/v3/groups/e9c9fbf316ac48fc9314c37c27e3f1ba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:59.340340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3abf2-8c2b-4795-9af0-fb3db85bf711 None admin] RBAC: Authorizing `identity:get_group(group_id=e9c9fbf316ac48fc9314c37c27e3f1ba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:59.341802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3abf2-8c2b-4795-9af0-fb3db85bf711 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:59.343464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 226/454] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 6 00:58:59 2022] GET /identity/v3/groups/e9c9fbf316ac48fc9314c37c27e3f1ba => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:58:59.349624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:58:59.452179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9785ead-18b8-4e34-b618-aa9ed12c43d3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:58:59.485559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c09f86-59bd-4b7b-97ef-b5cc53e594e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:58:59.486165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c09f86-59bd-4b7b-97ef-b5cc53e594e5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:58:59.486447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c09f86-59bd-4b7b-97ef-b5cc53e594e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:58:59.486725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c09f86-59bd-4b7b-97ef-b5cc53e594e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:58:59.487448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c09f86-59bd-4b7b-97ef-b5cc53e594e5 None admin] RBAC: Adding query filter params (group.id=e9c9fbf316ac48fc9314c37c27e3f1ba, role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=921fe46fc085463aa0816bc6ba906e3c) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:58:59.487841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c09f86-59bd-4b7b-97ef-b5cc53e594e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:58:59.489292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c09f86-59bd-4b7b-97ef-b5cc53e594e5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:58:59.495071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 229/455] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 6 00:58:59 2022] GET /identity/v3/role_assignments?group.id=e9c9fbf316ac48fc9314c37c27e3f1ba&scope.project.id=921fe46fc085463aa0816bc6ba906e3c&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:01.755080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c9f0-53c2-4723-80ff-6adef7ebe59a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:01.755573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c9f0-53c2-4723-80ff-6adef7ebe59a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:01.755680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c9f0-53c2-4723-80ff-6adef7ebe59a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:01.757106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 227/456] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:01.773196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e2ca-3b42-42a8-aaa8-6b34812b2803 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:01.774078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e2ca-3b42-42a8-aaa8-6b34812b2803 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:01.774546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e2ca-3b42-42a8-aaa8-6b34812b2803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:01.804777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e0c5e2ca-3b42-42a8-aaa8-6b34812b2803 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:01.806314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e0c5e2ca-3b42-42a8-aaa8-6b34812b2803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:01.818032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:01.892927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 230/457] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:02.119059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff70c2-43a4-4158-a638-454ecca13a50 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.119784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff70c2-43a4-4158-a638-454ecca13a50 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.120280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff70c2-43a4-4158-a638-454ecca13a50 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.122096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 228/458] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:02.128016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81ae0-d851-4e00-a2b5-8c643ce0d3bb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.128548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81ae0-d851-4e00-a2b5-8c643ce0d3bb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.129017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81ae0-d851-4e00-a2b5-8c643ce0d3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.156788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3a81ae0-d851-4e00-a2b5-8c643ce0d3bb None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:02.158258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3a81ae0-d851-4e00-a2b5-8c643ce0d3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:02.164703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c090d-cbcf-48be-829a-6b945e541156 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.165265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c090d-cbcf-48be-829a-6b945e541156 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.165738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c090d-cbcf-48be-829a-6b945e541156 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.167512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 229/459] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:02.172572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62219c-c3ec-42ba-87e7-d69af65542fa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.173118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62219c-c3ec-42ba-87e7-d69af65542fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.173590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62219c-c3ec-42ba-87e7-d69af65542fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.202744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6e62219c-c3ec-42ba-87e7-d69af65542fa None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:02.204324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6e62219c-c3ec-42ba-87e7-d69af65542fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:02.242114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 231/460] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:02.243335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81ae0-d851-4e00-a2b5-8c643ce0d3bb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:02.304981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 230/461] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:02.378037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:02.409294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81ae0-d851-4e00-a2b5-8c643ce0d3bb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:02.451409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:02.452094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.452370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.452640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.466155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:02.467926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:02.497482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:02.498524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:02.499890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:02.499890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:02.499890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:02.499890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 232/462] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:02.506309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62219c-c3ec-42ba-87e7-d69af65542fa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:02.614075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:02.665557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62219c-c3ec-42ba-87e7-d69af65542fa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:02.692110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:02.703884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d5d6f-f23d-474e-a004-017842c907e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:02.704487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d5d6f-f23d-474e-a004-017842c907e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.704770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d5d6f-f23d-474e-a004-017842c907e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.705031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d5d6f-f23d-474e-a004-017842c907e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.705662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d5d6f-f23d-474e-a004-017842c907e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:02.705955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d5d6f-f23d-474e-a004-017842c907e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:02.707319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d5d6f-f23d-474e-a004-017842c907e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:02.724104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 231/463] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:02.724104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95455520-c712-48c2-82bf-fbae327e53c6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.724104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95455520-c712-48c2-82bf-fbae327e53c6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.724104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95455520-c712-48c2-82bf-fbae327e53c6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.724104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 232/464] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:02.724104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95455520-c712-48c2-82bf-fbae327e53c6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:02.736695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83278f8-e046-4b63-80b9-3829ff1e4562 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:02.762982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:02.771385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:02.772045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.772322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.772775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.784071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:02.785527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:02.800825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:02.801780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:02.804854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:02.804854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:02.804854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:02.804854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 233/465] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:02.804854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:02.839102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95455520-c712-48c2-82bf-fbae327e53c6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:02.894298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62c863-71a2-46f1-ab21-8bec0ae10a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:02.894929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62c863-71a2-46f1-ab21-8bec0ae10a4d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.895221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62c863-71a2-46f1-ab21-8bec0ae10a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.895501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62c863-71a2-46f1-ab21-8bec0ae10a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.911066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62c863-71a2-46f1-ab21-8bec0ae10a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:02.912554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62c863-71a2-46f1-ab21-8bec0ae10a4d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5e62c863-71a2-46f1-ab21-8bec0ae10a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:02.931617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:02.932810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:02.933633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:02.933633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:02.933633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:02.933633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 233/466] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:02.933881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8840-c82a-4194-81b3-9c5a34e6981f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.934143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8840-c82a-4194-81b3-9c5a34e6981f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.934410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8840-c82a-4194-81b3-9c5a34e6981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.938411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2709a-c33b-4b55-8e46-3429ebfdc795 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:02.952292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c6fd8840-c82a-4194-81b3-9c5a34e6981f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:02.953473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6fd8840-c82a-4194-81b3-9c5a34e6981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:02.975990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:02.976607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:02.976892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:02.977172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:02.977848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:02.979408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:02.983784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:03.002640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 234/467] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:59:02 2022] POST /identity/v3/users => generated 246 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:03.003515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:03.040056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 234/468] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:03.041276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8840-c82a-4194-81b3-9c5a34e6981f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:03.065110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:03.195767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f1db8-20af-4375-ae77-cf75b50a5141 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:03.232126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8840-c82a-4194-81b3-9c5a34e6981f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:03.246275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:03.246891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.247164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.247613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.248121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:03.248405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:03.250682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:03.262767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 235/469] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:03.268824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:03.277644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:03.278549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.278974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.279396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.281277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:03.281785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:03.283853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:03.303986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 235/470] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:03.308313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:03.343071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:03.390340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:03.396767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db822998-1f4b-416a-9c53-3222d535138d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:03.439662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:03.440533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.440849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0ec2-7de7-4e8c-8ead-d39a8e814695 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:03.441135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.441618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.442560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:03.444959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:03.473261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 236/471] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:59:03 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:59:03.474361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:03.480441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351691f-264b-497c-9c46-acb1660e1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:03.481419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351691f-264b-497c-9c46-acb1660e1d78 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.481898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351691f-264b-497c-9c46-acb1660e1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.482370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351691f-264b-497c-9c46-acb1660e1d78 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.483279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351691f-264b-497c-9c46-acb1660e1d78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:03.485022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351691f-264b-497c-9c46-acb1660e1d78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:03.527608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 236/472] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:59:03 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:59:03.541249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f483-ba78-46aa-b532-2f2d5bb3dbb0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.541610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f483-ba78-46aa-b532-2f2d5bb3dbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.541818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f483-ba78-46aa-b532-2f2d5bb3dbb0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.543453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 237/473] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:03.549507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aac7bc-f100-4533-a1b3-d5f888033f47 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.550095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aac7bc-f100-4533-a1b3-d5f888033f47 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.550572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aac7bc-f100-4533-a1b3-d5f888033f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.573524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b4aac7bc-f100-4533-a1b3-d5f888033f47 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:03.575803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b4aac7bc-f100-4533-a1b3-d5f888033f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:03.674140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5033-e779-47e4-91e0-f63db79781ca None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:03.709401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 238/474] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:59:03.719495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:03.720217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.720498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.720772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.731971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:03.733442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:03.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:03.749691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:03.750526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:03.750526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:03.750526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:03.750526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 237/475] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:03.755624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aac7bc-f100-4533-a1b3-d5f888033f47 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:03.771177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:03.861718 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aac7bc-f100-4533-a1b3-d5f888033f47 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:03.896924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:03.897600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:03.897835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:03.898089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:03.898740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:03.899042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:03.900439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:03.912530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 239/476] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:03.917962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:03.923376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:04.055604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:04.071662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44bc95-0604-43ef-8dae-2d4ba21a8d83 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:04.132367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:04.132980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:04.133263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:04.133537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:04.134204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:04.135648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:04.176245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 238/477] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:59:03 2022] POST /identity/v3/users => generated 249 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:04.191103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:04.212283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b355-9917-4f02-8f8b-6709c334f940 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:04.267139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:04.268482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:04.271031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:04.271031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:04.271031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:04.275800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:04.299112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 00:59:04.299112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 00:59:04.299112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [parameters: {'id': 'e5a1c535a4fb4253b99324e03affc3b3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 00:59:04.299112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 06 00:59:04.316925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 00:59:04.317817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 00:59:04.318716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 00:59:04.319557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 00:59:04.320486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 00:59:04.321302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'id': 'e5a1c535a4fb4253b99324e03affc3b3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:04.322069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 00:59:04.324299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:04.326235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 240/478] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 6 00:59:04 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:04.331783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:04.398420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:04.444378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31282f-ced1-43c7-8027-d6313a41d0cd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:04.478718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a752944-620c-4ee7-8076-d612d98aebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:04.479355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752944-620c-4ee7-8076-d612d98aebb0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:04.479637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752944-620c-4ee7-8076-d612d98aebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:04.479958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752944-620c-4ee7-8076-d612d98aebb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:04.492343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a752944-620c-4ee7-8076-d612d98aebb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0a752944-620c-4ee7-8076-d612d98aebb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:04.497859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:04.499729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:04.500554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 239/479] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:04.505335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:04.620907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6d337-0ca1-4f6c-8cbe-857a5712aca7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:04.656147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eefbb9-8b8b-42f2-be60-7ca13306d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:04.656755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefbb9-8b8b-42f2-be60-7ca13306d8cb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:04.657019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefbb9-8b8b-42f2-be60-7ca13306d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:04.657289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefbb9-8b8b-42f2-be60-7ca13306d8cb None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:04.657941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eefbb9-8b8b-42f2-be60-7ca13306d8cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:04.658234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eefbb9-8b8b-42f2-be60-7ca13306d8cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:04.659598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eefbb9-8b8b-42f2-be60-7ca13306d8cb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:04.661859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:04.669570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 241/480] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:05.449774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deebf86-7caf-4949-b0a8-a8215272f3da None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:05.450313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deebf86-7caf-4949-b0a8-a8215272f3da None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:05.450313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deebf86-7caf-4949-b0a8-a8215272f3da None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:05.451967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 240/481] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:05.457283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9837a-c4e6-4dbe-9e05-59b65aafb006 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:05.457674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9837a-c4e6-4dbe-9e05-59b65aafb006 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:05.457840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9837a-c4e6-4dbe-9e05-59b65aafb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:05.484525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-90c9837a-c4e6-4dbe-9e05-59b65aafb006 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:05.485770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90c9837a-c4e6-4dbe-9e05-59b65aafb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:05.572484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 242/482] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:05.863472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8db1b-0b92-4c74-8043-6fd3da80325b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:05.863680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8db1b-0b92-4c74-8043-6fd3da80325b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:05.863971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8db1b-0b92-4c74-8043-6fd3da80325b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:05.865714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 241/483] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:05.871685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90da646-d284-4d4d-9aa2-cd884896cbce None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:05.872055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90da646-d284-4d4d-9aa2-cd884896cbce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:05.872334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90da646-d284-4d4d-9aa2-cd884896cbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:05.893041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a90da646-d284-4d4d-9aa2-cd884896cbce None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:05.894533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a90da646-d284-4d4d-9aa2-cd884896cbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:05.990301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 243/484] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:06.021068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8db1b-0b92-4c74-8043-6fd3da80325b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.038508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48254436-2fc9-445e-b740-486411f67562 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.039593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48254436-2fc9-445e-b740-486411f67562 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.040041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48254436-2fc9-445e-b740-486411f67562 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.042239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 244/485] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:06.047424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baabe9-1d60-4ba7-8968-56c497bd8019 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.047892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baabe9-1d60-4ba7-8968-56c497bd8019 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.048646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baabe9-1d60-4ba7-8968-56c497bd8019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.074544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-82baabe9-1d60-4ba7-8968-56c497bd8019 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:06.075716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-82baabe9-1d60-4ba7-8968-56c497bd8019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:06.145047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:06.159272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 245/486] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:06.215964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8db1b-0b92-4c74-8043-6fd3da80325b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:06.243625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:06.265507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:06.266144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.266425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.266705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.288907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:06.294003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:06.294874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:06.296127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 242/487] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:06.303052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baabe9-1d60-4ba7-8968-56c497bd8019 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.339283 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.441328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591204dd-75bb-43ef-9be2-f2fad2907e9c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:06.476490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:06.477098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.477495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.477767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.487953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baabe9-1d60-4ba7-8968-56c497bd8019 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:06.488938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:06.494439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:06.495349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:06.496173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 243/488] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:06.497438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.530513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:06.531207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.531485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.531848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.532502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:06.532902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:06.534426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:06.543378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 246/489] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:06.544495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:06.545333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.598584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:06.615413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4959029-d611-4831-9479-9b14241fc3b3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:06.652980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:06.653605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.653879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.654144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.656373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37086741-02ea-4cfc-8b2c-bdacd10cc66c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:06.669133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:06.676609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:06.677475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:06.678295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:06.681189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 244/490] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:06.682161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.700142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:06.700841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.701210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.702257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.703022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:06.703653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:06.705042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:06.719781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 247/491] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:06.719781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.800268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:06.804089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fe6d1-9eb8-4907-bf32-b6ae2f478666 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:06.840593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:06.841223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.841496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.841765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.852709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:06.854175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:06.855081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091f87-2d79-4be8-909f-4e2e2c7cfbb8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:06.869460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:06.872068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:06.875794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:06.875794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:06.875794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:06.875794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 245/492] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:06.876566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:06.876736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.894671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74251bf0-c1cd-437c-84cd-1161c0b037bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:06.895282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74251bf0-c1cd-437c-84cd-1161c0b037bf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.895562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74251bf0-c1cd-437c-84cd-1161c0b037bf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.895869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74251bf0-c1cd-437c-84cd-1161c0b037bf None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.896535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74251bf0-c1cd-437c-84cd-1161c0b037bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:06.896840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74251bf0-c1cd-437c-84cd-1161c0b037bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:06.898207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74251bf0-c1cd-437c-84cd-1161c0b037bf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:06.906384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 248/493] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:06.907555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37425dd3-a4d4-4188-b552-830a80865426 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:06.907870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37425dd3-a4d4-4188-b552-830a80865426 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:06.908139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37425dd3-a4d4-4188-b552-830a80865426 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:06.912276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 249/494] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:06.912276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37425dd3-a4d4-4188-b552-830a80865426 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:06.989478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c503-e671-46fe-9d51-972e284fe668 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.017994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37425dd3-a4d4-4188-b552-830a80865426 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.026244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.027090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.027513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.028071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.040572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.042219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.051192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d7df5-d30f-422c-95d0-e745422b5b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.051835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7df5-d30f-422c-95d0-e745422b5b48 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.052117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7df5-d30f-422c-95d0-e745422b5b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.052386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7df5-d30f-422c-95d0-e745422b5b48 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.053026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d7df5-d30f-422c-95d0-e745422b5b48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:07.053328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d7df5-d30f-422c-95d0-e745422b5b48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.054676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d7df5-d30f-422c-95d0-e745422b5b48 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:07.058685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:07.059776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:07.060818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:07.060818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:07.060818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:07.062788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 246/495] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:07.063921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.070562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.071017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 250/496] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:07.076495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8f0b-a1e9-49e8-a957-ba64574504c9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.076495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8f0b-a1e9-49e8-a957-ba64574504c9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.076495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f8f0b-a1e9-49e8-a957-ba64574504c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.092852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f60f8f0b-a1e9-49e8-a957-ba64574504c9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:07.094226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f60f8f0b-a1e9-49e8-a957-ba64574504c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:07.122403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.171821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 251/497] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:07.171821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f8f0b-a1e9-49e8-a957-ba64574504c9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.181634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e61287-91e3-4cc9-9bff-ffd4fa3d4735 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.218872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.219729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.220225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.220632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.233544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.235292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:07.250242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:07.251308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:07.252357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:07.252357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:07.252357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:07.258290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 247/498] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:07.258290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.286111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f8f0b-a1e9-49e8-a957-ba64574504c9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.317538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.320820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec371f-a6ac-48f2-8ff9-5090ca947372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.321424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec371f-a6ac-48f2-8ff9-5090ca947372 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.321702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec371f-a6ac-48f2-8ff9-5090ca947372 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.322028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec371f-a6ac-48f2-8ff9-5090ca947372 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.322669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec371f-a6ac-48f2-8ff9-5090ca947372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:07.322967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec371f-a6ac-48f2-8ff9-5090ca947372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.324353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec371f-a6ac-48f2-8ff9-5090ca947372 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.339863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 252/499] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:07.339863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391c6b-4cf7-4379-952c-5b4ed4e2ee2f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.339863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391c6b-4cf7-4379-952c-5b4ed4e2ee2f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.339863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391c6b-4cf7-4379-952c-5b4ed4e2ee2f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.339863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 253/500] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:07.339863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391c6b-4cf7-4379-952c-5b4ed4e2ee2f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.365968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.373402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709348e8-00b3-44e4-b792-7a6bd9b434cd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.408448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.409080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.409360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.409694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.417546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.418996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:07.432381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:07.433448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:07.434336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 248/501] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:07.434886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.450409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391c6b-4cf7-4379-952c-5b4ed4e2ee2f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.484745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d3327-221e-45d4-a372-7b3a1903ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.485337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d3327-221e-45d4-a372-7b3a1903ef7c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.485611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d3327-221e-45d4-a372-7b3a1903ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.485880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d3327-221e-45d4-a372-7b3a1903ef7c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.486511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d3327-221e-45d4-a372-7b3a1903ef7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:07.486805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d3327-221e-45d4-a372-7b3a1903ef7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.488187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d3327-221e-45d4-a372-7b3a1903ef7c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.498900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 254/502] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:07.500030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173563-7edd-4bc1-abbc-a4f0c69d6fef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.500367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173563-7edd-4bc1-abbc-a4f0c69d6fef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.500577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173563-7edd-4bc1-abbc-a4f0c69d6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.517761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e3173563-7edd-4bc1-abbc-a4f0c69d6fef None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:07.518923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3173563-7edd-4bc1-abbc-a4f0c69d6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:07.536101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb38c6-f013-4aa0-a72a-0f6ea68e9e75 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.552146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.569384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.570009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.570292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.570563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.578601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.580086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.589838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 255/503] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:07.590669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3173563-7edd-4bc1-abbc-a4f0c69d6fef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:07.591132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:07.592129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:07.594183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 249/504] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:07.596590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.606653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.699319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3173563-7edd-4bc1-abbc-a4f0c69d6fef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.710674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a328-6045-4bac-a268-123d755905f9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.738084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.738694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.738965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.739236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.739994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:07.740294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.741725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.746317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.747106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.747531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.747996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.758564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:07.765146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:07.766224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:07.767049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:07.769032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 250/505] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:07.769942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.778216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 256/506] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:07.779028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.812573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.856008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:07.881606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a87c24-b872-42bb-8598-7fcb6e253db5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.886370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a00820-7d0b-492c-a963-1b0aa97c0b03 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:07.915475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.916130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.916415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.916688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.921023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:07.921612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:07.921888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:07.922153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:07.922866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:07.923168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.924007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:07.924622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.925422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:07.936705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:07.937762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:07.938650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 251/507] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:07.939213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:07.945759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 257/508] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:07.946546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.033053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.039056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef500-70ba-4aa9-8d4c-619b1f55ce0e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.044375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0abc3-85fa-42de-9531-bc1caf5fcd4b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.071713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.072204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.072322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.072602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.072873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.073526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:08.073822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.075198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.078881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.079473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.079767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.080050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.083355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 252/509] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:08.084165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.146481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.147959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.182859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f379b-2a2a-40d7-bbc2-9663cedb1144 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.216262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.216876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.217159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.217563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.227818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.229267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:08.242876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:08.243730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:08.244613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:08.244613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.244613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:08.244772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 258/510] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:08.244772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.244900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 253/511] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:08.245878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.261659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.281357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.349951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6d51d-7a59-480b-bdcc-087bddfa23b9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.353025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46137-b64a-4a1d-8db4-e2e48020cfb5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.384098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.384690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.384965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.385230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.385937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:08.386234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.386767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.387366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.387645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.387797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.387969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.403720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 259/512] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:08.404518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.455771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.457225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.468496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.490891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.504015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995927d-7ac0-4b12-aeea-516125ebcd29 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.536631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.537251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.537535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.537810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.551518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.551518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.555270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 254/513] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:08.556152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:08.572582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:08.574074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:08.576103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:08.576103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.576103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:08.576103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 260/514] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:08.576103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.665955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e81e0-d8ca-4843-a7f5-8cdc3ee07445 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.674679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f6fed-12b7-44da-8611-658dec24228f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.676899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.700317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.700930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.701205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.701471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.707225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.707879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.708155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.708425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.709055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:08.709369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.710741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.711113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.712554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.719625 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.723230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 261/515] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:08.724261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:08.728041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:08.729074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:08.730127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:08.730127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.730127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:08.730127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 255/516] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:08.730127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.826771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62dc3d-e919-4385-8a9e-b6d6b15c0740 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.829167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a85677-a6cb-4058-bafd-9ff764d213f5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:08.862092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.862716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.862995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.863267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.867987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:08.868585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:08.868863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:08.869133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:08.873621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.875062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:08.888860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:08.889973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:08.890816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:08.890816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:08.890816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:08.890816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 256/517] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:08.890816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:08.907032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.939139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:08.941786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:08.943774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:08.991269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bd46a-16a7-4fdc-8ea4-5aeef9f10d4b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.024940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.025540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.025823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.026099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.026799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:09.027109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.028566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.040759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 257/518] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:09.040971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 262/519] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:09.041527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.041734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.117593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.134979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.138159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c78773-1411-4236-b9f4-67048afb3128 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.144938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9219814-0427-4045-868e-6d3b0a105e63 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.173079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.173674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.173953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.174224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.174854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:09.175154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.176541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.177791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.178378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.178657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.178925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.179555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.180987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.187459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 263/520] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:09.188275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.205495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 258/521] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:59:09 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:09.206266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.314340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be98b4e-3d1d-4a32-9bd9-2d52c72bd885 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.322624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132b9d9-0b86-4d53-a864-fec4e7ecd2b1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.354890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.355659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.355967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.356894 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.357256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:09.357573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.358982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.362673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.363325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.363899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.364181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.372890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 264/522] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:09.373032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.373975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.374347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:09.394450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:09.395398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:09.396380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 259/523] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:09.397420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.398360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.489014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868b6b8-b4c8-47f1-ba87-86f183ee9b4a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.504498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d1af5-2d1c-469a-9325-27db37fd234f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.525959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.526585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.526865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.527137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.537443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.538870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.541306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.541936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.542219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.542491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.549489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.550932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:09.553907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:09.554905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:09.556141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:09.556141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:09.556141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:09.559563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 265/524] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:09.560416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:09.563568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:09.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 260/525] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:09.565686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.626859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.651439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.682859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e094d-5952-4cc6-86f6-446b449d0c21 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.684961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd273ae8-715e-4d7b-b656-4d58eecbfd37 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.721514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.722217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.722595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.722872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.723059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.723598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:09.723716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.724040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.724123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.724260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.725343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.731026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.732491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:09.745219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:09.746081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:09.746915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 266/526] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:09.747447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.762399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 261/527] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:09.763235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.859556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af5f1-3b5d-4bc0-8573-6fde6b8ef41b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.875823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40288d2-239b-44a3-bc24-21eec2a5e96d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:09.887038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.891843 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:09.897954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.898556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.898852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.899123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.899778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:09.900091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.901469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.913512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:09.914122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 267/528] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:09.915079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:09.919244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:09.919244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:09.919458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:09.920145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:09.920940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:09.922452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:09.947370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 262/529] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:09.947370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.019074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b879f-069f-407c-9968-a283e77cb9bb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.050683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0de69-ef10-49ec-b8e1-145439faf23e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.055795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.056391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.056665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.056930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.057716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:10.058025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.059425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.077420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 268/530] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:10.078235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.094131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.094833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.095120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.095393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.130453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:10.140423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:10.164094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.165819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.191849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d4928-743e-4f09-9ccd-d30eb2d157ee None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.235062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.235688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.235991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.236277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.243154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.244634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:10.256055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:10.256905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:10.257761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:10.261850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 269/531] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:10.262599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.275439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 263/532] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:10.275439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.345860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:10.378886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087912-1027-4e00-bd05-be5a7ce39776 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.380449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870e007-811f-4063-9ecd-c70677821a84 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.393968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:10.415822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.416423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.416701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.416970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.417681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=1c737d4ca9ea488eb0aad2b328de193c) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:10.417788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.417987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.418337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.418616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.418884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.419416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.419591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=6b6caf1ad3784ddabda79ca0b1fc468d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:10.419924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.421999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.436088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 264/533] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/role_assignments?user.id=1c737d4ca9ea488eb0aad2b328de193c&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:10.436088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.445710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 270/534] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/role_assignments?user.id=6b6caf1ad3784ddabda79ca0b1fc468d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:10.446740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.645012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d4505-7e0c-44c9-89e4-a6df5aa3c31b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.656238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4ff3b-9db5-4f3a-9d25-71a73d5e0efe None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.708214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.708847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.709142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.709399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.710804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:10.724599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.725611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.725896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.726173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.726872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:10.727175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.727593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.728809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.729612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:10.744781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:10.745892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:10.746803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:10.746803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:10.746803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:10.748614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 271/535] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:10.750193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 265/536] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:10.756820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.758072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:10.824122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:10.891023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3b65e-6e3f-4a79-992a-3ca0c94be63e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.935296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:10.935950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:10.936760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:10.937037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:10.937867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:10.938174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:10.940253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:10.963670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efe846-0ad4-4373-9bd0-d7d579189756 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:10.973869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 272/537] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:10.979278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:11.018058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:11.018672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:11.018951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:11.019226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:11.019973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=10bbe5bd5324482b9a74836a552a077d) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:11.020286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:11.021734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:11.030849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 266/538] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/role_assignments?user.id=10bbe5bd5324482b9a74836a552a077d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:11.118594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:11.138006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb6b1e-67f9-497c-90ae-53587363cc05 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:11.189337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998d9c6-de87-45ef-bb29-a4d510b0e9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:11.189976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998d9c6-de87-45ef-bb29-a4d510b0e9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:11.190259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998d9c6-de87-45ef-bb29-a4d510b0e9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:11.190535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998d9c6-de87-45ef-bb29-a4d510b0e9c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:11.199040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998d9c6-de87-45ef-bb29-a4d510b0e9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:11.200596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998d9c6-de87-45ef-bb29-a4d510b0e9c0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6998d9c6-de87-45ef-bb29-a4d510b0e9c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:11.213379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:11.214219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:11.215058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 273/539] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:11.220302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:11.292114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:11.339616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ab7ce-b9a3-4cd5-928e-c424f4fb350f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:11.365935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6fd52-d236-41b9-b6fd-83202f9d21cb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:11.366214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6fd52-d236-41b9-b6fd-83202f9d21cb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:11.366487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6fd52-d236-41b9-b6fd-83202f9d21cb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:11.368208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 274/540] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:11.374719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:11.375331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:11.375607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:11.375914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:11.376613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:11.376922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:11.377087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62ad39-ecbf-4178-8d1c-0dd5017f75c6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:11.377371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62ad39-ecbf-4178-8d1c-0dd5017f75c6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:11.377656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62ad39-ecbf-4178-8d1c-0dd5017f75c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:11.378361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:11.395432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 267/541] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:11.398093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:11.418397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7d62ad39-ecbf-4178-8d1c-0dd5017f75c6 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:11.419613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7d62ad39-ecbf-4178-8d1c-0dd5017f75c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:11.506712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3680a7-4075-434f-971f-6b6346dbbcc0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:11.532218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:11.548843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:11.549511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:11.550515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:11.550515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:11.550815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=bc47f8d295ed481a87b4fb0d405c605e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:11.551104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:11.552627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:11.566906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 268/542] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:11 2022] GET /identity/v3/role_assignments?user.id=bc47f8d295ed481a87b4fb0d405c605e&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:11.585035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 275/543] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:11.917258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:12.055396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376e19-e4d8-497b-b748-4982042f1581 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:12.101332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:12.102320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.102813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.103273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.118697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:12.124342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:12.125198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:12.125981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:12.128534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 269/544] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:12.136868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62ad39-ecbf-4178-8d1c-0dd5017f75c6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:12.242218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:12.266269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62ad39-ecbf-4178-8d1c-0dd5017f75c6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:12.311233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:12.311874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.312157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.312436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.313104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:12.313407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:12.314790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:12.323453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 276/545] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:12.329889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:12.403613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:12.464193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457358b-8f51-4499-aeb6-6d876aa16ae0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:12.499915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ddd99-a2ff-447f-addc-18962061797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:12.500470 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ddd99-a2ff-447f-addc-18962061797c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.500749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ddd99-a2ff-447f-addc-18962061797c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.501022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ddd99-a2ff-447f-addc-18962061797c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.512768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ddd99-a2ff-447f-addc-18962061797c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:12.514825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ddd99-a2ff-447f-addc-18962061797c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-dd9ddd99-a2ff-447f-addc-18962061797c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:12.528998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:12.530452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:12.531261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:12.531261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:12.531261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:12.531261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 270/546] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:12.540861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:12.622141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd06532-d997-454c-af47-0ad8870b0b60 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.622753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd06532-d997-454c-af47-0ad8870b0b60 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.623247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd06532-d997-454c-af47-0ad8870b0b60 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.625094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 271/547] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:12.630028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1972-2c94-402c-add3-e94adc674a7f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.630544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1972-2c94-402c-add3-e94adc674a7f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.631016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1972-2c94-402c-add3-e94adc674a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.649635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-145c1972-2c94-402c-add3-e94adc674a7f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:12.651183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-145c1972-2c94-402c-add3-e94adc674a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:12.674314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594ff71-41de-4409-a93a-206f44470aec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:12.723078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:12.723698 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.724005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.724273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.724922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:12.725224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:12.726595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:12.744733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:12.750038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 277/548] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:12.760880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:12.768993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 272/549] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:59:12.817589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4522-6825-40d5-bf7c-71c63016029b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.818102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4522-6825-40d5-bf7c-71c63016029b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.818547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4522-6825-40d5-bf7c-71c63016029b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.820307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 273/550] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:12.824955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f18044-3e4a-4e81-adfe-b0f7ad83d6e9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.825396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f18044-3e4a-4e81-adfe-b0f7ad83d6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.825808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f18044-3e4a-4e81-adfe-b0f7ad83d6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:12.854970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a9f18044-3e4a-4e81-adfe-b0f7ad83d6e9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:12.856508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9f18044-3e4a-4e81-adfe-b0f7ad83d6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:12.876013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:12.928427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdaf75-889a-4986-8ee7-db733eeb7594 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:12.986173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 274/551] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:59:12.993250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:12.993895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:12.994171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:12.994451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.002009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:13.003535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:13.017040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:13.018101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:13.019024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 278/552] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:13.028827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f18044-3e4a-4e81-adfe-b0f7ad83d6e9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.073915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.140497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f18044-3e4a-4e81-adfe-b0f7ad83d6e9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.184704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.185836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.188225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.189294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.190481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:13.191806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:13.193597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:13.226235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 275/553] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:13.228107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:13.229148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.250125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:13.285589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3cfa0-81ae-47fd-942f-63e1ca329ffa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.333241 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.334303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.335242 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.335723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.350071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:13.357517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:13.358638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:13.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:13.360127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5887-72b9-43ce-ae10-71b1e9c28046 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.363078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 279/554] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:13.364126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.395626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.396270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.396573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.396875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.469364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:13.470658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:13.500191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbc057-3b13-4caf-a694-ba7bd55594c9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.520459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:13.523509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:13.540956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.541882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.542408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.542892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.554868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:13.560567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:13.562103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:13.562936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:13.565550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 280/555] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:13.566543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.585505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 276/556] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:13.586533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.676003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383df315-2b6b-4f6b-8a4d-ceb488ff5dd9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.694623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5976165-b52d-4308-bd27-124339a691ff None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.706124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:13.713990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.715202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.715714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.716288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.717153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:13.718101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:13.720011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:13.731456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 281/557] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:13.732133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.732803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.733439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.733679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.734199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.734613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:13.734933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:13.736788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:13.745831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 277/558] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:13.751597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.787085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:13.844752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c337ad-f4df-44a8-bb2b-755acd06d41f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.855800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb323dd0-63b5-4295-8966-45aae4472e91 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:13.882329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.882963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.883872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.883872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.891357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b39345-6a63-4d2e-ac42-433d3995614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:13.892022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39345-6a63-4d2e-ac42-433d3995614c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.892498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39345-6a63-4d2e-ac42-433d3995614c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.892626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39345-6a63-4d2e-ac42-433d3995614c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.897845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:13.899292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:13.903073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b39345-6a63-4d2e-ac42-433d3995614c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:13.904547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b39345-6a63-4d2e-ac42-433d3995614c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:13.913883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:13.915267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:13.916271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:13.916271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:13.916271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:13.916271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 282/559] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:13.917927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-42b39345-6a63-4d2e-ac42-433d3995614c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:13.923710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:13.924639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:13.925545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:13.925545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:13.925545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:13.925545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 278/560] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:13.926177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671f8a-5215-4a53-9560-c0db3905d77e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.926286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671f8a-5215-4a53-9560-c0db3905d77e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.926519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671f8a-5215-4a53-9560-c0db3905d77e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.927739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 279/561] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:13.931651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0522d-0811-447e-a7fe-099565ae2370 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:13.931651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0522d-0811-447e-a7fe-099565ae2370 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:13.931651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0522d-0811-447e-a7fe-099565ae2370 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:13.931651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 280/562] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:13.931651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0522d-0811-447e-a7fe-099565ae2370 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:13.955920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.018183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.024883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72f5ce-f812-4d7e-a017-4aeb35dfe513 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.032018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0522d-0811-447e-a7fe-099565ae2370 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.058931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c64c9-ab0c-4dd4-b61d-01ef8f012443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.059550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c64c9-ab0c-4dd4-b61d-01ef8f012443 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.059854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c64c9-ab0c-4dd4-b61d-01ef8f012443 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.060127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c64c9-ab0c-4dd4-b61d-01ef8f012443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.065104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.065686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.065965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.066231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.066860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:14.067160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.068824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.070818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c64c9-ab0c-4dd4-b61d-01ef8f012443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.072270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c64c9-ab0c-4dd4-b61d-01ef8f012443 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.080296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 281/563] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:14.081083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-042c64c9-ab0c-4dd4-b61d-01ef8f012443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:14.090158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:14.091135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:14.092070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:14.092070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:14.092070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:14.095943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 283/564] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:14.097275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63599b-507f-4687-a948-3dcbd065a66b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.097562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63599b-507f-4687-a948-3dcbd065a66b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.097840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63599b-507f-4687-a948-3dcbd065a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.115703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ce63599b-507f-4687-a948-3dcbd065a66b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:14.116890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce63599b-507f-4687-a948-3dcbd065a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:14.162108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.178712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4c6ff-40c5-4da2-bef4-0b50512efe0a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.183074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 284/565] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:14.184293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121a47d-90e7-49e5-b0f4-39128650e7a3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.184575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121a47d-90e7-49e5-b0f4-39128650e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.184845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121a47d-90e7-49e5-b0f4-39128650e7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.208181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a121a47d-90e7-49e5-b0f4-39128650e7a3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:14.209326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a121a47d-90e7-49e5-b0f4-39128650e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:14.212215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.212838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.213122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.213447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.214095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:14.214397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.215785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.232150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 282/566] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:14.232150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.290650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 285/567] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:14.292259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121a47d-90e7-49e5-b0f4-39128650e7a3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.311794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.346254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f38d7-9eeb-498c-aab2-7b2bc57294ba None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.381977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.382636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.382915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.383185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.391004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.392486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:14.404543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:14.406057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:14.406954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:14.410977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 283/568] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:14.411844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.416717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.431788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121a47d-90e7-49e5-b0f4-39128650e7a3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.472328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.472931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.473208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.473477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.474109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:14.474406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.475758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.486819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 286/569] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:14.487587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.522899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3e44b-7022-4fa6-be8f-44acec9ea48c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.556245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.560730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.561028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.561356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.561579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.569217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.570652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:14.582477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:14.583479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 284/570] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:14.584951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.599329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9a9a5-f5db-4ce9-b8b1-bf8f636de4aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.631267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.631890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.632169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.632442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.633123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:14.633422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.634836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.645847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.646768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 287/571] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:14.647597 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.689685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d33264-ce00-463a-b219-7f36d2e63aaf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.722342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.722964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.723244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.723515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.733589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:14.738794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:14.739789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:14.740647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 285/572] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:14.741344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.748355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd86d8-224d-4a0c-9b36-6144ee0b2d28 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.774292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.783587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.784232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.784514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.784788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.791762 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.793215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:14.804395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:14.805505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 288/573] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:14.806458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.843728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532df21-a60f-4991-b6d5-93bea80defc4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.866343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:14.878507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.878507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.878738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.878997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.889035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:14.894749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:14.895672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:14.896526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 286/574] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:14.897202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.911904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89930545-4659-43b9-9859-e0cd2e4c7d4c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:14.945666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:14.946262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:14.946543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:14.946817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:14.947530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:14.947860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:14.949274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:14.966085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 289/575] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:14.966863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:14.994109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3db43-3a89-4793-8f0d-204bcd838dba None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.013636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.027541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.028156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.028427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.028698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.029400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=e26430820d684ca8b4f63e473a16b6ca) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.029718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.031126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.036268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 287/576] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/role_assignments?user.id=e26430820d684ca8b4f63e473a16b6ca&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:15.037030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.073771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f284ff-cadf-44b9-8adf-a412dd1ee89b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.100655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.113782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.114497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.114747 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.114948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.115625 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.115987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.117445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.127126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 290/577] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:15.127803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.162653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7421c4-5731-4d6f-9959-b8bd87776973 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.202733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.204018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.204310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.204585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.205601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.205911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.208523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.226396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 288/578] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:15.227133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.243099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4299c3-4247-40b4-b245-6796722b3cf4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.251252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.282560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.283270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.283560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.283864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.284529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.284838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.286198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.294588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 291/579] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:15.300192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.335955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df77c21-e9cd-464c-a1d6-001e30273d30 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.343472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.374297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.374859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.375143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.375441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.386951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.388427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.403608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cfb89-aafa-42ff-9f19-d1cd036b4037 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:15.404068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:15.405727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:15.406685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:15.406685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.406685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:15.406685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 289/580] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:15.406991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.445118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.445752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.446035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.446308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.457459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.458914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:15.474564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:15.475729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:15.476837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:15.476837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.476837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:15.477532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 292/581] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:15.478473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.502931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.523331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0557a53-6ec6-48fe-8645-44cce1e004c7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.564851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.565499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.565775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.566042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.577466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.581086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:15.594564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:15.595661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:15.596591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:15.596591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.596591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:15.596591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 290/582] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:15.596877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.610501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.612134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bef2b-cc27-4229-94cb-9d42ef6a4de4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.648697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.649243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.649520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.649789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.660899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.662855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:15.680505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:15.681468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:15.682336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:15.682336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:15.682336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:15.682336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 293/583] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:15.682557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.710437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885cecc-8160-4b85-a2a0-242e11111326 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.746040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.746644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.747160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.747160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.747845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.748149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.749575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.756366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.762644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 291/584] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:15.762644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.791884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3c4d-b0b6-4a1e-818c-39cbdadc8075 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.826092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.827208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.827486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.827760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.828406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.828700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.830037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.839249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.841938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 294/585] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:15.843456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.876934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6943674-5dd3-48e0-8410-3fc2c46dc17e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.914822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.915666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.916161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.916628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.917459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.917980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.919401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:15.932115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 292/586] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:15.932922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:15.956360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b35e4-1b57-4646-b484-93fff878b275 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:15.988672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:15.994498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:15.995107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:15.995389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:15.995661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:15.996330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:15.996630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:15.998023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.014816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 295/587] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:16.014816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.041539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0d84b-15f2-4341-9c5b-78b5d42828c7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.078592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.085369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.086016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.086296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.086568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.095090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.096610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:16.109346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:16.110300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 293/588] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:16.111216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.123840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d0551-a174-488b-96ce-f01a98f54861 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.158413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.159435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.159820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.160138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.169997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.171515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:16.184498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:16.185423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:16.186875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 296/589] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:16.188174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.220714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c4f9-74cf-48ab-8a5c-45e8a659eba1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.244057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.260205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.260833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.261098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.261364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.268403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.269824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:16.281121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:16.281981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 294/590] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:16.282909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.299122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c2e46-e6f1-4c92-ad90-f1013e0a9328 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.333857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.334472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.334747 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.335012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.338352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.343300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.344796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:16.355941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:16.356890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 297/591] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:16.357868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.397565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1adee4-509d-43e6-9fa5-ff21612494a6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.432602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.433205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.433598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.433871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.434557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:16.434857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.437037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.454996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 295/592] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:16.455989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.457201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c4010-af8f-425b-bf88-486c9b4ef51d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.470500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.493920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.494727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.494974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.495333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.496165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:16.496562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.498026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.522389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 298/593] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:16.522389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.569194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e0110-391c-4d61-ac1e-ca6ce7c84cb6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.583371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.609475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.610083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.610362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.610632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.611314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:16.611644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.625423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 296/594] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:16.628617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.636466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401f583-0fee-4ff6-b57a-e0d89f70fc71 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.673631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.674342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.674697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.675035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.675858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:16.676228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.677743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.695903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 299/595] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:16.696662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.727192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.735885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711c02c-a15d-4224-840b-2001366d99ad None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.772490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.773233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.773512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.773780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/1c737d4ca9ea488eb0aad2b328de193c/roles/b26b251c03cd4a2eac5f85927c38b797` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.789297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=1c737d4ca9ea488eb0aad2b328de193c, role_id=b26b251c03cd4a2eac5f85927c38b797)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.791019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.803419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e291b-3903-40d1-97e0-0e9a0f81983d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.804622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 297/596] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:16 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/1c737d4ca9ea488eb0aad2b328de193c/roles/b26b251c03cd4a2eac5f85927c38b797 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:59:16.805580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.843709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.846958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.847703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.848110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.848417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:16.862923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:16.865207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:16.883188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:16.884294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:16.885310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:16.885310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:16.885310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:16.889446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 300/597] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:16.890272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:16.947239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab510cf3-b83f-4d5b-9a40-3e6c09bac767 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:16.989362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:16.991127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:16.991903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:16.992242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:16.992521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/6b6caf1ad3784ddabda79ca0b1fc468d/roles/b26b251c03cd4a2eac5f85927c38b797` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.007757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=6b6caf1ad3784ddabda79ca0b1fc468d, role_id=b26b251c03cd4a2eac5f85927c38b797)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.009401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.012265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d69b74-8327-4620-bd3d-471536430d47 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.024046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 298/598] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:16 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/6b6caf1ad3784ddabda79ca0b1fc468d/roles/b26b251c03cd4a2eac5f85927c38b797 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:59:17.025404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.051078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4a170-4822-4759-a08d-9dae6e03ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.051929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4a170-4822-4759-a08d-9dae6e03ba67 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.052300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4a170-4822-4759-a08d-9dae6e03ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.052666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4a170-4822-4759-a08d-9dae6e03ba67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.065915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4a170-4822-4759-a08d-9dae6e03ba67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.067628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4a170-4822-4759-a08d-9dae6e03ba67 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-26b4a170-4822-4759-a08d-9dae6e03ba67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:17.083582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:17.084853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:17.085989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:17.085989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:17.085989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:17.087385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 301/599] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:17.088870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94ebba-411a-4ac1-9523-01ad20e47f5c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.089252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94ebba-411a-4ac1-9523-01ad20e47f5c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.089615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94ebba-411a-4ac1-9523-01ad20e47f5c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.095792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 302/600] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:17.095792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94ebba-411a-4ac1-9523-01ad20e47f5c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.126087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:17.172051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5819c1-2d6c-4903-aa4d-6e66e450b50c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.214659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfe02e-9e11-4c15-8cc6-a4139ac76223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.215352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe02e-9e11-4c15-8cc6-a4139ac76223 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.216035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe02e-9e11-4c15-8cc6-a4139ac76223 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.216550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe02e-9e11-4c15-8cc6-a4139ac76223 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.217966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfe02e-9e11-4c15-8cc6-a4139ac76223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:17.218382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfe02e-9e11-4c15-8cc6-a4139ac76223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.220427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfe02e-9e11-4c15-8cc6-a4139ac76223 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.234773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 299/601] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:17.236232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c439c-1305-4927-a20a-991efdc8e3b2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.236622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c439c-1305-4927-a20a-991efdc8e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.236995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c439c-1305-4927-a20a-991efdc8e3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.243668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94ebba-411a-4ac1-9523-01ad20e47f5c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.255152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dc0c439c-1305-4927-a20a-991efdc8e3b2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:17.256507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dc0c439c-1305-4927-a20a-991efdc8e3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:17.286292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:17.293425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.294209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.294883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.295306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.297076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:17.297532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.299038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.314252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 303/602] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:17.315296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.335281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 300/603] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:17.336713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c439c-1305-4927-a20a-991efdc8e3b2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.420647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:17.425756 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13f8fd-f182-4d89-8c8a-d537063e52e7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.462674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.463667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.463840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.464157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.472264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.473686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.474409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c439c-1305-4927-a20a-991efdc8e3b2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:17.485952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:17.487022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:17.488091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:17.491332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 304/604] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:17.493113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.518654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.519783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.520612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.521056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.530787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.533257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:17.545021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:17.546118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:17.547006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:17.549105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 301/605] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:17.558031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.585077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:17.685046 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974392e-ed1b-45df-a58c-257919cfc9f5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.727415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777c57c-c5d3-49e9-9d42-269e9e2a3e50 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.735726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.736558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.736848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.737228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.738019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:17.738443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.739988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.753230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:17.758659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 305/606] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:17.759700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.770566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.771871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.772333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.772752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.774053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:17.774509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.776641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:17.796284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 302/607] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:17.797321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.897197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022a1a8-2898-4b57-b6e6-b9abf0ca8c64 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.915730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:17.917670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11600b-e618-4dfb-8631-4a0b335f2287 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:17.947260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.947929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.948215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.948631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.959643 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:17.960527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:17.960955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:17.961371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/10bbe5bd5324482b9a74836a552a077d/roles/b26b251c03cd4a2eac5f85927c38b797` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:17.961484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:17.967567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:17.968731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:17.969664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 306/608] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:17.970384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:17.985487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=10bbe5bd5324482b9a74836a552a077d, role_id=b26b251c03cd4a2eac5f85927c38b797)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:17.987187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:18.005368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 303/609] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:17 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/10bbe5bd5324482b9a74836a552a077d/roles/b26b251c03cd4a2eac5f85927c38b797 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:59:18.006434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:18.043276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:18.092706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af74505-155f-4611-97ee-0099e759fbfa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:18.128301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5442f4-cab2-416c-a813-ee84b0093b19 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:18.132632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:18.133316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:18.133592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:18.133865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/bc47f8d295ed481a87b4fb0d405c605e/roles/b26b251c03cd4a2eac5f85927c38b797` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:18.151992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=bc47f8d295ed481a87b4fb0d405c605e, role_id=b26b251c03cd4a2eac5f85927c38b797)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:18.153592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:18.173466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:18.174838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:18.175777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:18.177213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:18.178085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:18.178781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:18.182841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:18.185428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 307/610] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:17 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/bc47f8d295ed481a87b4fb0d405c605e/roles/b26b251c03cd4a2eac5f85927c38b797 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:59:18.194901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 304/611] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:18.200355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:18.232273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:18.335642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635e632-2214-4e41-8a5e-8586fcaeaf28 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:18.379050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:18.379701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:18.380010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:18.380288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:18.392359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:18.393949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:18.410844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:18.411971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:18.413881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:18.413881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:18.413881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:18.413881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 308/612] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:18.418233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:18.520665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:18.530760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185ea81-161e-41b7-b372-6728198b4fe1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:18.575308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:18.576340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:18.576888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:18.577366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:18.578652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:18.579229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:18.580940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:18.595677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 305/613] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:18.604823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:18.714898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:18.749748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585c86-921c-4be1-8599-bdd31398f44f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:18.799310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38b4b9-7f8d-4bf5-99b0-942b61fbfa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:18.799971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38b4b9-7f8d-4bf5-99b0-942b61fbfa04 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:18.800255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38b4b9-7f8d-4bf5-99b0-942b61fbfa04 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:18.800533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38b4b9-7f8d-4bf5-99b0-942b61fbfa04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:18.808360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38b4b9-7f8d-4bf5-99b0-942b61fbfa04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:18.809805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38b4b9-7f8d-4bf5-99b0-942b61fbfa04 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ec38b4b9-7f8d-4bf5-99b0-942b61fbfa04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:18.823194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:18.824240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:18.826216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 309/614] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:18.830840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:18.910073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e601f63-a479-496b-9e62-c468b2eb2bd2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:18.910380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e601f63-a479-496b-9e62-c468b2eb2bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:18.910552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e601f63-a479-496b-9e62-c468b2eb2bd2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:18.912275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 310/615] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:18.917644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032159-a708-4666-b9da-6f25889c1572 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:18.917925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032159-a708-4666-b9da-6f25889c1572 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:18.918195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032159-a708-4666-b9da-6f25889c1572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:18.936917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-41032159-a708-4666-b9da-6f25889c1572 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:18.938096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41032159-a708-4666-b9da-6f25889c1572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:18.973892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddfc0d-de1c-4061-90e9-6bb000c4f535 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:19.016660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621501-1204-4873-8928-f80bf306451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:19.018025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621501-1204-4873-8928-f80bf306451c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.019041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621501-1204-4873-8928-f80bf306451c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.020654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621501-1204-4873-8928-f80bf306451c None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.021603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61621501-1204-4873-8928-f80bf306451c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:19.023054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61621501-1204-4873-8928-f80bf306451c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:19.030899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61621501-1204-4873-8928-f80bf306451c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:19.065536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 306/616] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 6 00:59:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:19.071327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621501-1204-4873-8928-f80bf306451c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:19.090546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 311/617] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:59:19.094619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:19.214073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621501-1204-4873-8928-f80bf306451c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:19.249920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:19.250551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.250828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.251097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.261345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:19.262801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:19.278591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:19.280282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:19.281317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:19.281317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:19.281317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:19.281317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 307/618] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:19.285336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032159-a708-4666-b9da-6f25889c1572 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:19.392473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:19.422959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:19.446266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032159-a708-4666-b9da-6f25889c1572 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:19.495850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:19.495850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.495850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.495850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.496224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:19.496224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:19.496224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:19.506796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 312/619] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:19.516826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:19.536176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:19.573062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29afb-dd2e-4a1c-a07b-d566cbde9267 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:19.610232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2071f-f560-4ad4-8bc6-4306452fcbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:19.611145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2071f-f560-4ad4-8bc6-4306452fcbbc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.611652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2071f-f560-4ad4-8bc6-4306452fcbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.613006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2071f-f560-4ad4-8bc6-4306452fcbbc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.635085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ac537-d43e-47c1-806a-ef74b9879a52 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:19.640713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2071f-f560-4ad4-8bc6-4306452fcbbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cef2071f-f560-4ad4-8bc6-4306452fcbbc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:19.648168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:19.649248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:19.650518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 308/620] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:19.651521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafc78f-d1e4-40ad-8868-008473769f88 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.652030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafc78f-d1e4-40ad-8868-008473769f88 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.652030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafc78f-d1e4-40ad-8868-008473769f88 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.654181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 309/621] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:19.657088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafc78f-d1e4-40ad-8868-008473769f88 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:19.664719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:19.676702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b0f61-cf2c-4d56-a0c4-62eda9af07c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:19.677774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b0f61-cf2c-4d56-a0c4-62eda9af07c9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.678203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b0f61-cf2c-4d56-a0c4-62eda9af07c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.678581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b0f61-cf2c-4d56-a0c4-62eda9af07c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.685873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b0f61-cf2c-4d56-a0c4-62eda9af07c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:19.688016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b0f61-cf2c-4d56-a0c4-62eda9af07c9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e86b0f61-cf2c-4d56-a0c4-62eda9af07c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:19.701522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:19.702649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:19.703745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:19.709984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 313/622] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:19.709984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4248eb-2808-4f61-be10-3383c682cfc5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.709984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4248eb-2808-4f61-be10-3383c682cfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.709984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4248eb-2808-4f61-be10-3383c682cfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.725713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ee4248eb-2808-4f61-be10-3383c682cfc5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:19.727036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ee4248eb-2808-4f61-be10-3383c682cfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:19.799963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafc78f-d1e4-40ad-8868-008473769f88 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:19.810420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 314/623] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:19.812218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4248eb-2808-4f61-be10-3383c682cfc5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:19.824865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:19.837106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:19.837695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:19.838051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:19.838430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:19.839159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:19.839558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:19.841110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:19.849911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 310/624] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:19.857200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:19.962476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4248eb-2808-4f61-be10-3383c682cfc5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:19.977744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:20.008419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.013713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.014220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.014643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.015480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:20.015951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.017580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:20.033354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 315/625] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:20.055293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6333d-a667-478f-bb7d-e66733e78fef None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.057588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:20.127296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.127933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.128216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.128491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.150176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.151575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:20.167968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032a90e-a5e5-4132-b161-7e0d9c11d832 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:20.178060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:20.179196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:20.184349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:20.184349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:20.184349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:20.184349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 311/626] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:20.193481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:20.211255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.212204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.212629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.213045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/e26430820d684ca8b4f63e473a16b6ca/roles/b26b251c03cd4a2eac5f85927c38b797` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.229691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=e26430820d684ca8b4f63e473a16b6ca, role_id=b26b251c03cd4a2eac5f85927c38b797)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.231435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:20.246978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:20.251811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 316/627] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:20 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/e26430820d684ca8b4f63e473a16b6ca/roles/b26b251c03cd4a2eac5f85927c38b797 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 00:59:20.252959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:20.317292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:20.387053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c612-3e05-438c-ae03-d1e8beaaad7d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.430911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45158168-adc0-493c-a486-b002faef512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.431541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45158168-adc0-493c-a486-b002faef512d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.431858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45158168-adc0-493c-a486-b002faef512d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.432124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45158168-adc0-493c-a486-b002faef512d None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.432778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45158168-adc0-493c-a486-b002faef512d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:20.433082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45158168-adc0-493c-a486-b002faef512d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.434468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45158168-adc0-493c-a486-b002faef512d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:20.453766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 312/628] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:20.465696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45158168-adc0-493c-a486-b002faef512d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:20.480198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80ac2b-9d12-48af-b4d8-19cadac4838a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.524905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7c464-72bf-4427-9297-f1d3cec44b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.525988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7c464-72bf-4427-9297-f1d3cec44b1d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.526369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7c464-72bf-4427-9297-f1d3cec44b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.526738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7c464-72bf-4427-9297-f1d3cec44b1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.543651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7c464-72bf-4427-9297-f1d3cec44b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7bb7c464-72bf-4427-9297-f1d3cec44b1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:20.550157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:20.551259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:20.552352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:20.559373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 317/629] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:20.559373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fc648-4d9f-4699-8953-8846289a0a7e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.559373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fc648-4d9f-4699-8953-8846289a0a7e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.559373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fc648-4d9f-4699-8953-8846289a0a7e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.559373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 318/630] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:20.563425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fc648-4d9f-4699-8953-8846289a0a7e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:20.600149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:20.609338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45158168-adc0-493c-a486-b002faef512d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.649921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436a6f4-c863-4c1a-821b-09fd26b3dfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.650560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a6f4-c863-4c1a-821b-09fd26b3dfa2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.650840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a6f4-c863-4c1a-821b-09fd26b3dfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.651329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a6f4-c863-4c1a-821b-09fd26b3dfa2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.659130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436a6f4-c863-4c1a-821b-09fd26b3dfa2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.660620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436a6f4-c863-4c1a-821b-09fd26b3dfa2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f436a6f4-c863-4c1a-821b-09fd26b3dfa2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:20.673248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:20.674241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:20.675165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 313/631] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:20.675774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c64656-1b2f-4ab3-83e5-ae63f2e96353 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.675774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c64656-1b2f-4ab3-83e5-ae63f2e96353 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.677773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c64656-1b2f-4ab3-83e5-ae63f2e96353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.688835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:20.695848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b8c64656-1b2f-4ab3-83e5-ae63f2e96353 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:20.697186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b8c64656-1b2f-4ab3-83e5-ae63f2e96353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:20.711561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fc648-4d9f-4699-8953-8846289a0a7e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.746566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.747194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.747469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.747725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.748408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:20.750151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.751505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:20.768014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 319/632] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:20.768014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:20.783038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 314/633] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:20.783038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284b5f0-15ab-487e-81db-3202e4e15a31 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.783038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284b5f0-15ab-487e-81db-3202e4e15a31 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.783038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284b5f0-15ab-487e-81db-3202e4e15a31 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.783038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 315/634] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:20.783038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284b5f0-15ab-487e-81db-3202e4e15a31 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:20.895520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:20.907563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe994-9b49-49a7-b035-0f369927034e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.924683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284b5f0-15ab-487e-81db-3202e4e15a31 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:20.943383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9114ab-a666-4fbf-996f-9a786770bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.944043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9114ab-a666-4fbf-996f-9a786770bd15 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.944317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9114ab-a666-4fbf-996f-9a786770bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.944651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9114ab-a666-4fbf-996f-9a786770bd15 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.945546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9114ab-a666-4fbf-996f-9a786770bd15 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:20.945734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9114ab-a666-4fbf-996f-9a786770bd15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.947107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9114ab-a666-4fbf-996f-9a786770bd15 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:20.965957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 320/635] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 6 00:59:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:20.967041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c530c93-bfa5-497a-a4f9-9b6ed684be88 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.967447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c530c93-bfa5-497a-a4f9-9b6ed684be88 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.967645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c530c93-bfa5-497a-a4f9-9b6ed684be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.972255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:20.973184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:20.973728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:20.974499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:20.976198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:20.988537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1c530c93-bfa5-497a-a4f9-9b6ed684be88 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:20.989755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c530c93-bfa5-497a-a4f9-9b6ed684be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:20.992105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:20.994679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:21.015893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:21.016867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:21.017773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:21.017773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:21.017773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:21.026139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 316/636] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:21.026139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.087777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 321/637] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:21.087777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c530c93-bfa5-497a-a4f9-9b6ed684be88 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.152523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8d543-7d94-4b0f-89fa-d26a02f2c278 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.193939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c530c93-bfa5-497a-a4f9-9b6ed684be88 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.195625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.197099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.197452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.197786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.208114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:21.231313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.232066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.232396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.232675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.233856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:21.234158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:21.235775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.237231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:21.255387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 322/638] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:21.256258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.283345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:21.284985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.362109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff917ef-d238-4c55-b75b-88fb314dbef3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.396961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.397589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.397866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.398138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.408770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:21.410278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 317/639] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:21.411285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:21.414472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:21.415661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 323/640] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:21.416534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.465899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:21.478209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:21.526162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22721414-8480-474f-aedc-59c105d38bc7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.530887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544b56d-590b-4efe-8dbd-0ab7d7ba47bb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.563330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.564259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.564683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.565099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.566770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.567382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.567657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.567953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.574792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:21.576268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.581866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:21.587899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:21.588803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 318/641] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:21.589719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:21.591411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:21.592981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:21.593832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 324/642] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:21.594533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.696655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:21.701515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447f034-00bd-4db3-8099-17dbd8005f92 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.707455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c9f2-3880-4675-9203-d9ec2eaf7f0a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.722751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:21.739145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.739791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.740084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.740339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.745860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.746515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.746790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.747082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.748182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:21.748512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:21.750520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.751434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:21.752928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.758530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 325/643] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:21.760168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:21.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:21.769093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:21.775282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:21.775282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:21.775282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:21.775282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 319/644] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:21.775282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.871446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d7e4d-98df-4a5a-a23d-0654ffb97d6f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.878000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27654966-8e3c-4563-938b-4ecaea871685 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:21.906924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.907512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.908094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.908805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:21.909105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:21.910508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.913172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:21.913785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:21.914063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:21.914334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:21.915245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:21.915547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:21.916947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:21.930334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 320/645] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:21.931190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.933559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 326/646] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:21.934640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:21.952906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:21.984550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.062222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae9c8-4cca-4094-af22-8d06165ad762 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.065628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc44519-0194-49c5-b6fd-d2ec229e7fb8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.103778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.104763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.105239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.105705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.106239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.106780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:22.106885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.107138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.107410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.107789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.109384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.119458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.121801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.125506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 327/647] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:22.126435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:22.145255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:22.146290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:22.147167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:22.147167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:22.147167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:22.151336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 321/648] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:22.152547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.228713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.246830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.255251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9e5d5-eeb6-4a06-a31e-f9ee291090e3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.277140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc50388-84df-496c-ab07-fd74ffd415b4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.295832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.296554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.296906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.297249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.309808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.312022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.313200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.313960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.314072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.314339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:22.326369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:22.327977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:22.328847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:22.328847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:22.328847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:22.333043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 328/649] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:22.333043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.380451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.381898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.428789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b40cbf-56b4-49f0-a298-05c3677cbac3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.440272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.462614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb988fd-ae2a-4e29-aba6-3d1b66af6217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.463228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb988fd-ae2a-4e29-aba6-3d1b66af6217 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.463502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb988fd-ae2a-4e29-aba6-3d1b66af6217 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.463797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb988fd-ae2a-4e29-aba6-3d1b66af6217 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.470201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb988fd-ae2a-4e29-aba6-3d1b66af6217 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.471616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb988fd-ae2a-4e29-aba6-3d1b66af6217 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.477663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 322/650] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:22.478489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ebb988fd-ae2a-4e29-aba6-3d1b66af6217 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:22.483713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:22.484763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:22.485862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 329/651] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:22.486434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cfeab-05e8-46cd-9e2e-cbb810a86bb9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.486562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cfeab-05e8-46cd-9e2e-cbb810a86bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.486837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cfeab-05e8-46cd-9e2e-cbb810a86bb9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.491785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 330/652] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:22.491785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cfeab-05e8-46cd-9e2e-cbb810a86bb9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.498304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.582261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c18abd-29c9-4903-8ace-9083c96e11a6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.593592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cfeab-05e8-46cd-9e2e-cbb810a86bb9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.615551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.616178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.616457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.616730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.617363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:22.617663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.619048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.627467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.628084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.628361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.628630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.629252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:22.629547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.630732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 323/653] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:22.630881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.631619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.646019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 331/654] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:22.646019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.657702 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.705162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.730883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e79aa-7b75-433e-82d2-5c072bd0d5b0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.739115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f147db-9d61-49fa-88b0-a020567415bd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.763625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.764350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.764629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.764928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.771229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472e3c9-9d0f-40f4-828e-339f29414fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.771866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472e3c9-9d0f-40f4-828e-339f29414fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.772144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472e3c9-9d0f-40f4-828e-339f29414fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.772410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472e3c9-9d0f-40f4-828e-339f29414fb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.773080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472e3c9-9d0f-40f4-828e-339f29414fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:22.773376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472e3c9-9d0f-40f4-828e-339f29414fb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.774783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472e3c9-9d0f-40f4-828e-339f29414fb9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.775055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.776505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.787158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 332/655] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:22.788428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cd273-0444-4771-9b7f-76e4003cd658 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.788808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cd273-0444-4771-9b7f-76e4003cd658 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.789171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cd273-0444-4771-9b7f-76e4003cd658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:22.790476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:22.791570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:22.792762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:22.792762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:22.792762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:22.796047 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 324/656] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:22.797077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.808932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d24cd273-0444-4771-9b7f-76e4003cd658 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:22.810098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d24cd273-0444-4771-9b7f-76e4003cd658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:22.871193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.874353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 333/657] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:22.875169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cd273-0444-4771-9b7f-76e4003cd658 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.910352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143d446-c5c0-41c7-9c86-081fec94ee17 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:22.931828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:22.945982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:22.946613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:22.946896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:22.947306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:22.954618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:22.956086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:22.967701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:22.968689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 325/658] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:22.969647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:22.982532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cd273-0444-4771-9b7f-76e4003cd658 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.018352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.018911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.019189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.019460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.026518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.027982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:23.040417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:23.041317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 334/659] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:23.042276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.078585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586340d-23a8-4f8a-94dd-3cb25caa7972 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.113497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.114100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.114378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.114648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.115366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=1c737d4ca9ea488eb0aad2b328de193c) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:23.115669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.117133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.119979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.124576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 326/660] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:22 2022] GET /identity/v3/role_assignments?user.id=1c737d4ca9ea488eb0aad2b328de193c&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:23.125384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.157580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bbc6f-f97c-4b51-8db8-abeb63a4f3f8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.176649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.197071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.197806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.198086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.198361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.199025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:23.199328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.200786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.218504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 335/661] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:23.218504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.246437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b017bc-0acd-45bc-81e4-c59fffddc171 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.286155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.286776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.287115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.287388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.288142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:23.288447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.290116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.312145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 327/662] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:23.313151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.332873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4f72-3e40-4caf-b072-a352cc63ee4b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.368203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.368905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.369278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.369630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.370426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:23.370797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.372328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.378592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.391335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 336/663] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:23.392059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.428039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.430171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9436753-bf8a-4b1b-b2c1-d6f8bc811bf8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.465943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.466666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.466945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.467263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.478555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:23.483893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:23.484802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:23.485665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 328/664] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:23.486385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.509192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d83e8-2eab-4d69-b8a9-936704c9369a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.546694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.547260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.547526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.552041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.559091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.560958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:23.571900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:23.573033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:23.574199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:23.580126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 337/665] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:23.581911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.607625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0108c-1559-43db-9ab6-e96917470c53 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.627080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.646846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.647565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.647870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.648142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.676606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.686856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01406182-7fa3-467d-87ef-b5254eafcdbc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.721730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.723376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.723979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.724592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.724866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.725137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.795211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.799208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:23.839737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 329/666] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:23.840865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.860563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.897522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:23.906642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 338/667] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:23.907444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:23.950427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1d416-0a32-4721-88da-9cb0db20c26a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:23.985432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:23.986037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:23.986312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:23.986580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:23.987238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:23.987536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:23.988949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.002053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 330/668] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:24.002053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.015583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0992a-a8db-463d-9582-db6a131ed9be None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.049994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.050611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.050964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.051305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.052090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:24.052473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.053964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.071053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 339/669] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:24.071984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.085744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.122523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae506558-e3e0-4552-ba19-cc1fd6a6525d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.136496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.159959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.160591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.160871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.161138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.172427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.173882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.184599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c7d45-513d-4f07-abde-da94b867a7ac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:24.188407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:24.189554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:24.190655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:24.190655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:24.190655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:24.194108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 331/670] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:24.194705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.223074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.223866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.224249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.224626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.237703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.239281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:24.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:24.256640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:24.257611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:24.257611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:24.257611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:24.257611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 340/671] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:24.257846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.301184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bb707-e751-410f-806f-d7419a64c8c0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.332772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.339184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.339974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.340658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.340933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.352323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.353758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.366480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5049c5-f010-432e-be69-c0eadecb0020 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:24.368461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:24.369564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:24.370649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:24.370649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:24.370649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:24.374907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 332/672] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:24.375513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.377397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.413651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.414576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.415489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.416754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.417646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=6b6caf1ad3784ddabda79ca0b1fc468d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:24.418113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.419675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.428689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 341/673] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/role_assignments?user.id=6b6caf1ad3784ddabda79ca0b1fc468d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:24.429884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.495906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee98029-cc46-40d3-88b2-c2a9fa903a79 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.543443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.544188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.544370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.544714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.545290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:24.546210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.547724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.557599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642a76d-d1f0-4324-bebf-69d822ff7f22 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.565979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 333/674] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:24.565979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.610880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.611491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.611803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.612063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.612701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:24.612996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.614348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.630033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 342/675] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:24.631463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.634954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.669157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.714338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49019289-8f0e-47be-8197-1ec37cc39549 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.755196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d374c-5fb2-4559-8360-c674826a4a06 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.764768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.766069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.766519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.767347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.768461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:24.768971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.771344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.789323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 334/676] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:24.790822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.792521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.793402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.793745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.794084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.801617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.803138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:24.814981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:24.816096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:24.817197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:24.818383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 343/677] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:24.819337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.920065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621383ad-d275-4e5f-baff-a68622c40ba2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.923356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.927373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:24.941380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d8994-34ca-4e88-857e-16cc5c5ad9b9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:24.962807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.964113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.964565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.965001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.975563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.978222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:24.982765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:24.983535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:24.983940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:24.984514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:24.992764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:24.993863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:24.994910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:24.995916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:24.996593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 335/678] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:24.998558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:24.998691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:25.016624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:25.017630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:25.018690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:25.025981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 344/679] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:25.025981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:25.112217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5830b-1e7a-41b2-9eca-bd63fdbf6ae1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:25.147482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08371f7e-8991-48a5-a895-49dd25837e8c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:25.153746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:25.155557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.156088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.156581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.157468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:25.157954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:25.159553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.172436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 336/680] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:25.173250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:25.188263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fbb29-3491-4758-8050-865541f52b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:25.189656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fbb29-3491-4758-8050-865541f52b86 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.190153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fbb29-3491-4758-8050-865541f52b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.190531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fbb29-3491-4758-8050-865541f52b86 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.191289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fbb29-3491-4758-8050-865541f52b86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:25.191606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fbb29-3491-4758-8050-865541f52b86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:25.193108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fbb29-3491-4758-8050-865541f52b86 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.199300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:25.211016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 345/681] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:25.211016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cae5c-6e3e-443a-bad4-8c39307a1b4b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.211016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cae5c-6e3e-443a-bad4-8c39307a1b4b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.211016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cae5c-6e3e-443a-bad4-8c39307a1b4b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.211016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 346/682] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:25.211016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cae5c-6e3e-443a-bad4-8c39307a1b4b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:25.223283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:25.286771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecb5cf-616d-47da-9714-0ee6fce8c8d6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:25.321022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cae5c-6e3e-443a-bad4-8c39307a1b4b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:25.323218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:25.323846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.324129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.324397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.325115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:25.325415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:25.327628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.346093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 337/683] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:25.347119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:25.363203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97e5ee-b6c2-40dc-8c4b-9a3a8716fd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:25.364029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97e5ee-b6c2-40dc-8c4b-9a3a8716fd3c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.364310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97e5ee-b6c2-40dc-8c4b-9a3a8716fd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.364582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97e5ee-b6c2-40dc-8c4b-9a3a8716fd3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.365296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97e5ee-b6c2-40dc-8c4b-9a3a8716fd3c None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=10bbe5bd5324482b9a74836a552a077d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:25.365599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97e5ee-b6c2-40dc-8c4b-9a3a8716fd3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:25.366999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97e5ee-b6c2-40dc-8c4b-9a3a8716fd3c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.372649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 347/684] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/role_assignments?user.id=10bbe5bd5324482b9a74836a552a077d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:25.374309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37078f57-39f6-4ae9-a78a-248d570db53b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.374588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37078f57-39f6-4ae9-a78a-248d570db53b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.374857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37078f57-39f6-4ae9-a78a-248d570db53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.394043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-37078f57-39f6-4ae9-a78a-248d570db53b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:25.395264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-37078f57-39f6-4ae9-a78a-248d570db53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:25.454569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:25.476619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 348/685] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:25.484014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91c7bd-4852-46f9-aa7e-5c327167ba2f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:25.485200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37078f57-39f6-4ae9-a78a-248d570db53b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:25.491875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:25.525881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:25.526661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.527038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.527404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.528521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=bc47f8d295ed481a87b4fb0d405c605e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:25.528934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:25.530655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.544627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 338/686] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/role_assignments?user.id=bc47f8d295ed481a87b4fb0d405c605e&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:25.660731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37078f57-39f6-4ae9-a78a-248d570db53b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:25.703302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:25.703953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.704234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.704503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.777416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:25.786767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:25.788854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.792854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:25.917940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92954aa-3f0e-4e1c-8c96-74a02715bd94 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:25.926303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 349/687] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:25.931723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:25.961927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:25.962971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:25.963255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:25.963801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:25.964399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:25.966031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:25.986245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:25.987321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:25.988442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 339/688] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:25.994609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:26.010873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:26.099099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6054-d0cc-4274-96f4-cae13641eeb7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:26.105668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:26.119578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d5f6-f1ac-4281-b6ab-7a240349fc55 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:26.160924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:26.161554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.161829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.162093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.183141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:26.184623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:26.193133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:26.193743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.194014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.194280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.195021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:26.195322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:26.196723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:26.206094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 340/689] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:26.215034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:26.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:26.218090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:26.218090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:26.218090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:26.218090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 350/690] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:26.219495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:26.227774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:26.336837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:26.352873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea68a-40ca-47cd-b10e-7d58c1bd53f6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:26.386630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8476507-ba19-4ebc-90c3-21e02cd3c7be None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:26.387079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:26.387674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.387994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.388268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.388912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:26.389735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:26.391097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:26.403508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 351/691] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:26.410216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:26.441485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:26.460226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608aecaf-55f5-4e6a-b6ac-bcbbb6aca6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:26.460853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aecaf-55f5-4e6a-b6ac-bcbbb6aca6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.461131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aecaf-55f5-4e6a-b6ac-bcbbb6aca6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.461402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aecaf-55f5-4e6a-b6ac-bcbbb6aca6e8 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.462038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608aecaf-55f5-4e6a-b6ac-bcbbb6aca6e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:26.462336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608aecaf-55f5-4e6a-b6ac-bcbbb6aca6e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:26.463707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608aecaf-55f5-4e6a-b6ac-bcbbb6aca6e8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:26.480381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 341/692] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 6 00:59:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:26.556422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e06c4-5901-4be7-b00e-393720602064 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:26.602969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:26.603620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.603933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.604210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.611502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:26.613067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:26.624564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9c933-9bcc-410f-923a-c7aa339e4351 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.625089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9c933-9bcc-410f-923a-c7aa339e4351 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.625166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9c933-9bcc-410f-923a-c7aa339e4351 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:26.625603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:26.626422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:26.627997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:26.628438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 342/693] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:26.628507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 352/694] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:26.632244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37b7fb-66c8-4ae2-9953-eb81bdf86b92 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.632531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37b7fb-66c8-4ae2-9953-eb81bdf86b92 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.632705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:26.632861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37b7fb-66c8-4ae2-9953-eb81bdf86b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.680981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ac37b7fb-66c8-4ae2-9953-eb81bdf86b92 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:26.682269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ac37b7fb-66c8-4ae2-9953-eb81bdf86b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:26.726083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:26.751139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7534922-2c48-4e6b-be6e-8753810732c3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:26.770608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 343/695] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:26.788701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:26.789311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.789595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.790123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.790817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:26.791122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:26.792582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:26.805647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 353/696] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:26.812459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37b7fb-66c8-4ae2-9953-eb81bdf86b92 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:26.912112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:26.954433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37b7fb-66c8-4ae2-9953-eb81bdf86b92 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:26.988844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:26.989501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:26.989877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:26.990151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:26.991055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=e26430820d684ca8b4f63e473a16b6ca) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:26.991360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:26.992921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:26.999797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 344/697] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:26 2022] GET /identity/v3/role_assignments?user.id=e26430820d684ca8b4f63e473a16b6ca&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:27.043639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:27.161371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b563f-9d47-4a4a-a4ee-edf1e14b3bc0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:27.207278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:27.207949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:27.208233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:27.208507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:27.214021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:27.224043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:27.228934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:27.229863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:27.231068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 354/698] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:27.240915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:27.350472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24370c13-aac3-46d9-9902-c06548bb647c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:27.380495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:27.390000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:27.390615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:27.390895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:27.391162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:27.391860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:27.392162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:27.393530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:27.403657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 345/699] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:27.410763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:27.524645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d176f2e-5bbe-428b-872f-fedae66550ec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:27.571242 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:27.571911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:27.572194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:27.572468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:27.587951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:27.587951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:27.612469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:27.613665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:27.614800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:27.614800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:27.614800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:27.616196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 355/700] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:27.624843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:27.789265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430cc86-ba41-49c8-a574-c9ee3d11f11c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:27.847819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86317e-f2a3-45d6-b405-5372b20b870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:27.848750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86317e-f2a3-45d6-b405-5372b20b870c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:27.849185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86317e-f2a3-45d6-b405-5372b20b870c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:27.849616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86317e-f2a3-45d6-b405-5372b20b870c None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:27.850314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86317e-f2a3-45d6-b405-5372b20b870c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:27.850620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86317e-f2a3-45d6-b405-5372b20b870c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:27.852043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86317e-f2a3-45d6-b405-5372b20b870c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:27.872883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 346/701] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:27.877982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:27.923777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3678-8cff-4430-bd49-db47121596d4 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:27.923777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3678-8cff-4430-bd49-db47121596d4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:27.923777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3678-8cff-4430-bd49-db47121596d4 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:27.924684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 347/702] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:27.929620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccace8-324b-48a7-9e4f-1806ea073d0f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:27.929875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccace8-324b-48a7-9e4f-1806ea073d0f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:27.930146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccace8-324b-48a7-9e4f-1806ea073d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:27.952381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:27.964750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8eccace8-324b-48a7-9e4f-1806ea073d0f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:27.966962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8eccace8-324b-48a7-9e4f-1806ea073d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:28.024187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61da34-b6ff-4db9-b566-4ad9e37804df None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:28.066018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9e064-a81a-4efb-a6f2-4512c0150048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:28.066589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e064-a81a-4efb-a6f2-4512c0150048 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.066873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e064-a81a-4efb-a6f2-4512c0150048 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.067152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e064-a81a-4efb-a6f2-4512c0150048 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.072178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:28.087366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9e064-a81a-4efb-a6f2-4512c0150048 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:28.088849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9e064-a81a-4efb-a6f2-4512c0150048 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:28.093575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 348/703] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-48f9e064-a81a-4efb-a6f2-4512c0150048 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:28.117489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:28.118488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:28.119456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 356/704] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 00:59:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:28.124324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccace8-324b-48a7-9e4f-1806ea073d0f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:28.156072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e064-fbd4-4ac7-b56c-786fac453d44 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.157009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e064-fbd4-4ac7-b56c-786fac453d44 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.157009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e064-fbd4-4ac7-b56c-786fac453d44 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.158239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 357/705] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:28.165861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783e42a-48db-4e05-ac5d-942c33dfe998 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.166152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783e42a-48db-4e05-ac5d-942c33dfe998 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.166526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783e42a-48db-4e05-ac5d-942c33dfe998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.206151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3783e42a-48db-4e05-ac5d-942c33dfe998 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:28.207443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3783e42a-48db-4e05-ac5d-942c33dfe998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:28.245612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccace8-324b-48a7-9e4f-1806ea073d0f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:28.281113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:28.282033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.282507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.283000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.283953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:28.284489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:28.286155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:28.304136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 349/706] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 00:59:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:28.311819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:28.352566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 358/707] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:28.384125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:28.469426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd80d99-b7a9-486f-a2f8-42904be18790 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:28.490505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783e42a-48db-4e05-ac5d-942c33dfe998 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:28.532823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:28.533441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.533725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.533996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.565041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:28.614704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:28.616253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:28.628494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783e42a-48db-4e05-ac5d-942c33dfe998 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:28.665513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:28.666155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.666428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.666711 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.667297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:28.668844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:28.683250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:28.684364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:28.685267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:28.692790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 359/708] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:28 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:28.693657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:28.696536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:28.763239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 350/709] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:28.764141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:28.821674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63890a8-9011-48e0-8055-bb7c548c19d4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:28.823204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:28.874117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:28.874679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.874874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623d15f-968f-4a0e-a4fc-8c38d184f2a7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:28.875034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.875218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.889598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:28.891881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:28.911888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:28.912194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:28.914339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:28.916727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:28.916727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:28.916727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:28.917119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:28.917119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:28.917119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:28.917529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 360/710] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:28.917529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:28.921749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:28.921749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:28.921749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:28.926288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 351/711] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:28.927087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:28.938422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:29.037507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34dcbc-77d9-448f-9d56-6d0ec37fa2e7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:29.062146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2bf39-68d1-4d32-8616-65dc75db7e71 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:29.074953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:29.075561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.075874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.076144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.076825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:29.077121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:29.078484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:29.090162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 352/712] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:29.091252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:29.107321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e0a60-9b75-44a3-8bb3-9852105499f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:29.108926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e0a60-9b75-44a3-8bb3-9852105499f0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.109219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e0a60-9b75-44a3-8bb3-9852105499f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.109491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e0a60-9b75-44a3-8bb3-9852105499f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.123955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e0a60-9b75-44a3-8bb3-9852105499f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:29.125444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e0a60-9b75-44a3-8bb3-9852105499f0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ee3e0a60-9b75-44a3-8bb3-9852105499f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:29.139682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:29.140654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:29.141564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:29.141564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:29.141564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:29.141564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 361/713] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:29.142267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7868a-4789-434c-8dda-1c583a8375c4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.142267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7868a-4789-434c-8dda-1c583a8375c4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.142519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7868a-4789-434c-8dda-1c583a8375c4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.147613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 362/714] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:29.147613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7868a-4789-434c-8dda-1c583a8375c4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:29.155331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:29.166698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:29.204795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832949a-3d8b-4aa6-9f60-055c59219c03 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:29.239055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:29.239654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.239958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.240227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.240843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] RBAC: Adding query filter params (type=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:29.241139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:29.242490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:29.255788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 353/715] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:29.255788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:29.267010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7868a-4789-434c-8dda-1c583a8375c4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:29.306099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139d133-360f-4b88-bfb8-ea079e3500e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:29.306814 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d133-360f-4b88-bfb8-ea079e3500e0 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.307156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d133-360f-4b88-bfb8-ea079e3500e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.307494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d133-360f-4b88-bfb8-ea079e3500e0 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.308259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139d133-360f-4b88-bfb8-ea079e3500e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:29.309794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139d133-360f-4b88-bfb8-ea079e3500e0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:29.339635 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 363/716] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 00:59:29 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:29.341909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ba17e-1406-41ac-8693-3c6c2e007751 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.343233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ba17e-1406-41ac-8693-3c6c2e007751 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.343576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ba17e-1406-41ac-8693-3c6c2e007751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.368541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-149ba17e-1406-41ac-8693-3c6c2e007751 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:29.369898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-149ba17e-1406-41ac-8693-3c6c2e007751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:29.395781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190bc3c-a37c-464a-91df-b8abf9af4668 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:29.455622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:29.466195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:29.467069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.468040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.468315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.468976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:29.469277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:29.470684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:29.482719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:29.497646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 354/717] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:29.498988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 364/718] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:29.499266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:29.550588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152344-6c7d-43ad-8b03-d4913f1d9b13 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.551047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152344-6c7d-43ad-8b03-d4913f1d9b13 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.551439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152344-6c7d-43ad-8b03-d4913f1d9b13 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.553190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 365/719] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:29.558231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4073e2-a850-4265-aca6-f1e956f38861 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.558621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4073e2-a850-4265-aca6-f1e956f38861 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.559003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4073e2-a850-4265-aca6-f1e956f38861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.577795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8f4073e2-a850-4265-aca6-f1e956f38861 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:29.579147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f4073e2-a850-4265-aca6-f1e956f38861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:29.625513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3565a71-cd01-4b5e-87d8-baab68f2fc89 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:29.666828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:29.667842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.668699 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.669191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.674786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 366/720] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:59:29.680914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:29.682606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:29.695544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:29.696587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:29.697885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:29.703871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 355/721] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:29.707085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4073e2-a850-4265-aca6-f1e956f38861 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:29.812802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4073e2-a850-4265-aca6-f1e956f38861 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:29.829007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:29.852942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:29.853601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:29.853883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:29.854152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:29.854860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:29.855166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:29.856614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:29.871812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 367/722] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:29.872116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:29.875727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:29.921854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:30.013903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1007ae-d13f-414f-a85c-41ab5b3bd917 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:30.041170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f750b-096a-4387-b3fc-52a7f1132077 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:30.050304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:30.051010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:30.051365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:30.051714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:30.052474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:30.053940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:30.072435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 356/723] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:59:29 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:59:30.073502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:30.094153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:30.094819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:30.095106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:30.095377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:30.096122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=6b6caf1ad3784ddabda79ca0b1fc468d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:30.096431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:30.097885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:30.104839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 368/724] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:29 2022] GET /identity/v3/role_assignments?user.id=6b6caf1ad3784ddabda79ca0b1fc468d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:30.202028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f940459-a82c-446f-b586-558d2e094010 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:30.214555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:30.244924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:30.245567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:30.245960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:30.246243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:30.258679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:30.264285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:30.266492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:30.268690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 357/725] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:30.271805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:30.322743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:30.389960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8c513-67fa-42c2-9f35-964848ad0e85 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:30.436079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:30.436699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:30.436975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:30.437242 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:30.437900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:30.438198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:30.439569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:30.448879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 369/726] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:30.454354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:30.601972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec88c2-5a21-4a45-b01d-b6997e0e7d44 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:30.657061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:30.658064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:30.658588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:30.659079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:30.677672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:30.679426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:30.696720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:30.698111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:30.699000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:30.699000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:30.699000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:30.700813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 358/727] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:30.707051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:30.997839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed51013-0260-434d-8b84-79ed6513cf9f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:31.001304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:31.049014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:31.049673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:31.049889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:31.050147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:31.050839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:31.051142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:31.052525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:31.070801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 370/728] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:31.075875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:31.126705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:31.240779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a277d75-2aab-49ed-abb0-fa28f5219f87 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:31.291132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:31.291684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:31.291985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:31.292253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:31.300636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:31.302077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:31.313387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:31.314312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:31.315241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 359/729] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:31.320054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:31.491396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3623c-04ad-4a96-b0e0-d249c95c4a2e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:31.531745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efa6f0-241b-4d4c-8ff6-e913b775d691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:31.532385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efa6f0-241b-4d4c-8ff6-e913b775d691 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:31.532662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efa6f0-241b-4d4c-8ff6-e913b775d691 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:31.532930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efa6f0-241b-4d4c-8ff6-e913b775d691 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:31.533653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efa6f0-241b-4d4c-8ff6-e913b775d691 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:31.533956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efa6f0-241b-4d4c-8ff6-e913b775d691 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:31.535422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efa6f0-241b-4d4c-8ff6-e913b775d691 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:31.555528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 371/730] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 6 00:59:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:31.561024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:31.719831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:31.732598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff524-0a13-4f18-a374-dcf8a02721d6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:31.733050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff524-0a13-4f18-a374-dcf8a02721d6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:31.733480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff524-0a13-4f18-a374-dcf8a02721d6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:31.734995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 372/731] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:31.744038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e9725-4f62-4511-9527-16071b256315 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:31.749110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a6eb-ee55-4143-9bf8-fb9d0f3a3182 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:31.749394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a6eb-ee55-4143-9bf8-fb9d0f3a3182 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:31.749664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a6eb-ee55-4143-9bf8-fb9d0f3a3182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:31.787370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cfd18-58cf-4726-bcbd-ac18683789db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:31.788015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cfd18-58cf-4726-bcbd-ac18683789db None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:31.788295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cfd18-58cf-4726-bcbd-ac18683789db None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:31.788568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cfd18-58cf-4726-bcbd-ac18683789db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:31.796795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-90d6a6eb-ee55-4143-9bf8-fb9d0f3a3182 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:31.798015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90d6a6eb-ee55-4143-9bf8-fb9d0f3a3182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:31.849325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:31.856968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cfd18-58cf-4726-bcbd-ac18683789db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:31.858975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cfd18-58cf-4726-bcbd-ac18683789db None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:31.940111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:31.977145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 373/732] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 00:59:31.977411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 360/733] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 00:59:31.989308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6a6eb-ee55-4143-9bf8-fb9d0f3a3182 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:32.036961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a814f-9908-4d92-8665-f49d939640d8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.037305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a814f-9908-4d92-8665-f49d939640d8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.037508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a814f-9908-4d92-8665-f49d939640d8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.038841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 361/734] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:32.044079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a01206-b22a-4726-b35a-e537a84c1012 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.044356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a01206-b22a-4726-b35a-e537a84c1012 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.044624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a01206-b22a-4726-b35a-e537a84c1012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.069001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-18a01206-b22a-4726-b35a-e537a84c1012 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:32.072471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-18a01206-b22a-4726-b35a-e537a84c1012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:32.169987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6a6eb-ee55-4143-9bf8-fb9d0f3a3182 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:32.210260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 362/735] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 00:59:32.237133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:32.237774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.238051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.238322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.255418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:32.256956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:32.283554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:32.284587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:32.285499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:32.285499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:32.285499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:32.285681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 374/736] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:32.290364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a01206-b22a-4726-b35a-e537a84c1012 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:32.300721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:32.311582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:32.444142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a01206-b22a-4726-b35a-e537a84c1012 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:32.458106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:32.462130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3863c-d9e6-4586-a171-2696058bbef3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:32.482341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6bf81-da68-4487-90a1-301dbe4f0b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:32.483580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6bf81-da68-4487-90a1-301dbe4f0b69 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.483956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6bf81-da68-4487-90a1-301dbe4f0b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.484621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6bf81-da68-4487-90a1-301dbe4f0b69 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.485703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6bf81-da68-4487-90a1-301dbe4f0b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:32.486093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6bf81-da68-4487-90a1-301dbe4f0b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:32.488221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6bf81-da68-4487-90a1-301dbe4f0b69 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:32.501570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 363/737] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:32.502562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82b520-21bc-40a3-b900-13cdc2494fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:32.502892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4835d0b-6080-4836-bc5d-9638c975689a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.503173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4835d0b-6080-4836-bc5d-9638c975689a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.503392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82b520-21bc-40a3-b900-13cdc2494fa0 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.503493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4835d0b-6080-4836-bc5d-9638c975689a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.503807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82b520-21bc-40a3-b900-13cdc2494fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.504158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82b520-21bc-40a3-b900-13cdc2494fa0 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.504882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82b520-21bc-40a3-b900-13cdc2494fa0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:32.504990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 364/738] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:32.506933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4835d0b-6080-4836-bc5d-9638c975689a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:32.510776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82b520-21bc-40a3-b900-13cdc2494fa0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:32.527273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 375/739] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:59:32 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:32.528849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f60ca-652f-40ad-aa27-db28b7103c53 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.529260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f60ca-652f-40ad-aa27-db28b7103c53 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.529629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f60ca-652f-40ad-aa27-db28b7103c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.562129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d77f60ca-652f-40ad-aa27-db28b7103c53 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:32.563771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d77f60ca-652f-40ad-aa27-db28b7103c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:32.640193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4835d0b-6080-4836-bc5d-9638c975689a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:32.646082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:32.659857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 376/740] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:32.661206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f60ca-652f-40ad-aa27-db28b7103c53 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:32.679064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2346c8-4b04-431d-a232-cda466dbf729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:32.687841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2346c8-4b04-431d-a232-cda466dbf729 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.687841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2346c8-4b04-431d-a232-cda466dbf729 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.687841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2346c8-4b04-431d-a232-cda466dbf729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.697212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2346c8-4b04-431d-a232-cda466dbf729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:32.698628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2346c8-4b04-431d-a232-cda466dbf729 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fd2346c8-4b04-431d-a232-cda466dbf729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:32.711212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:32.712261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:32.713350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 365/741] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:32.713942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7f04e-0b14-4bff-8079-0b50784d852b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.714086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7f04e-0b14-4bff-8079-0b50784d852b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.714366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7f04e-0b14-4bff-8079-0b50784d852b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.715905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 366/742] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:32.721366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a416-62e2-4c07-99e3-d33e97d47f55 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.722306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a416-62e2-4c07-99e3-d33e97d47f55 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.722876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a416-62e2-4c07-99e3-d33e97d47f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.741497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-89c8a416-62e2-4c07-99e3-d33e97d47f55 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:32.742688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-89c8a416-62e2-4c07-99e3-d33e97d47f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:32.766324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:32.790512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f60ca-652f-40ad-aa27-db28b7103c53 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:32.815645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 367/743] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:32.816650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8a416-62e2-4c07-99e3-d33e97d47f55 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:32.835048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:32.835998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:32.836488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:32.836938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:32.849967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:32.855539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:32.856499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:32.857340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:32.860736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 377/744] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:32.867713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:32.950716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:32.963361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8a416-62e2-4c07-99e3-d33e97d47f55 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:32.999516 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.000295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.000645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.000985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.001549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eea6-a488-44b1-a599-1e4c7a7ad6d2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.001798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:33.002172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.003700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.016266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 368/745] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:33.021572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.037354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:33.048616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.049482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.050038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.050529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.051523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:33.052023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.053565 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.062937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 378/746] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:33.064204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.123811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb909c1-03ae-4780-9c7d-f3519716e769 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.160742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.161377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.161656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.161926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.172749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:33.175866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881b767-66f5-4f6e-ab7f-49e5038fdaaa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:33.178751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:33.184289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 369/747] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:33.185326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.194076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:33.213698 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.214236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.214518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.214798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.215510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=e26430820d684ca8b4f63e473a16b6ca) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:33.215848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.217825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.228126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 379/748] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/role_assignments?user.id=e26430820d684ca8b4f63e473a16b6ca&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:33.229992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.274473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:33.301681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03726055-9d3f-4098-97d6-20fe947a82d8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.339994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.340533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.340812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.341084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.354021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.355536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:33.372238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:33.373448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:33.374432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:33.374432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:33.374432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:33.374432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 370/749] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:33.375228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.383678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d48c-b814-4412-a617-0d41ec86add7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.430107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.430737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.431014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.431285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.431889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.433246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:33.442768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:33.443796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:33.444643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 380/750] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:33.445441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.493930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:33.516466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43555829-120b-47d6-ba0b-4382f93023c6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.552546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.553144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.553408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.553683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.554345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:33.554647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.556061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.565668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 371/751] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:33.566560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.566790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e887870-8a8b-4d8d-8833-9bb5db3f5e13 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.584182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:33.611634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.612617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.613078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.613657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.614486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:33.614980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.616575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.634024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 381/752] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:33.635217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.690005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76172c39-7c6f-409b-855d-32f86365fed2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.735709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.736620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.736959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.737238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.738063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:33.738373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.740071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.746811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 372/753] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:33.747818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.761052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc9e8f-2233-4ab9-a03c-b8cc38c8cf6c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.763707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:33.807985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.809072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.809360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.809634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.824897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:33.825249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.827829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:33.850139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:33.851145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:33.852456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:33.852456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:33.852456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:33.852456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 382/754] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:33.852949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.877515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289782c3-5a02-4a45-9edb-4ac396ed75f2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:33.918545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:33.919624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:33.920162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:33.921168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:33.929843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:33.931561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:33.944582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:33.945551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:33.946473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:33.948995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 373/755] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:33.950018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:33.994115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcee9fc-cc83-4239-b909-0073a978579e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.030377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.030910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.031194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.031466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.032133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:34.032875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.033899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.046232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 383/756] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:34.046232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.068001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.084369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a87fe3-178a-4a5c-8c69-de6feb6cdc93 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.104486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.130731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.131498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.131915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.132319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.133298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:34.133602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.135569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.151475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 374/757] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:34.152714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.158434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1193c-ef0e-431f-8cb3-8f6856708581 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.195891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.197344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.197842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.198382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.199384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:34.199966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.201695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.225701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 384/758] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:34.226947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.262166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d6244-f99f-4ac7-a117-1853c4dbb6df None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.304294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.304834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.305141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.305417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.306115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] RBAC: Adding query filter params (interface=public, service_id=d2dd0e25a3ce446795207058fcd04754, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:34.306423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.307863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.318769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.320990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 375/759] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/endpoints?service_id=d2dd0e25a3ce446795207058fcd04754&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:34.321940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.347103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555968a-5ef2-4098-9ab8-fca38e9d29c4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.367834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.385249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.386182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.386769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.387246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.395122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.396840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:34.409890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:34.410868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:34.411897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:34.413953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 385/760] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:34.415226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.430118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237f46f-65dd-4c9c-8fd8-248cc6e810cb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.464541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c89d6-ba22-4ed9-ad51-493e4a268f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.465138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c89d6-ba22-4ed9-ad51-493e4a268f15 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.465419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c89d6-ba22-4ed9-ad51-493e4a268f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.465697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c89d6-ba22-4ed9-ad51-493e4a268f15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.524298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92490cf-349b-46c4-901d-0a970fed1a72 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.536575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c89d6-ba22-4ed9-ad51-493e4a268f15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.538044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c89d6-ba22-4ed9-ad51-493e4a268f15 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.561496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.562465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.562928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.563380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.564197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.565786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.571538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 386/761] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:34.572543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.575307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.595402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.698856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da02acf-39f9-4441-9e8b-b45489ab0317 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.705905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 376/762] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 00:59:34.711617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed376d-d6e3-4244-8b68-18399e6bd839 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.711617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed376d-d6e3-4244-8b68-18399e6bd839 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.711617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed376d-d6e3-4244-8b68-18399e6bd839 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.714077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 377/763] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:34.715025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed376d-d6e3-4244-8b68-18399e6bd839 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.756125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3e9d5-e263-45bc-ae1f-66c47c10d259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.757480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3e9d5-e263-45bc-ae1f-66c47c10d259 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.757480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3e9d5-e263-45bc-ae1f-66c47c10d259 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.757480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3e9d5-e263-45bc-ae1f-66c47c10d259 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.758461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3e9d5-e263-45bc-ae1f-66c47c10d259 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:34.758461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3e9d5-e263-45bc-ae1f-66c47c10d259 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.760382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3e9d5-e263-45bc-ae1f-66c47c10d259 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.784050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 387/764] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:34.784646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db591-3538-46b1-acc7-cc108c031323 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.784931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db591-3538-46b1-acc7-cc108c031323 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.785259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db591-3538-46b1-acc7-cc108c031323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.805149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-248db591-3538-46b1-acc7-cc108c031323 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:34.806293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-248db591-3538-46b1-acc7-cc108c031323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:34.838490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed376d-d6e3-4244-8b68-18399e6bd839 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:34.866802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.878072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:34.878980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:34.879401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:34.879969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:34.887777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 388/765] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:34.887777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248db591-3538-46b1-acc7-cc108c031323 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.896135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:34.901564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:34.903048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:34.935112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:34.936897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:34.937816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:34.937816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:34.937816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:34.941200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 378/766] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:34.946321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:34.994267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248db591-3538-46b1-acc7-cc108c031323 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.030909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.031644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.032024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.032371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.054589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.057270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:35.076311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:35.077832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:35.078710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:35.078710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:35.078710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:35.078710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 389/767] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:35.083977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.140277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:35.149670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dca66-7bf9-435e-81f3-ce46b37f10db None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.196438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fc4bf-ed42-4861-ac22-eba1ef622c69 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.205312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a2b5f-2254-4936-a1a1-873e6fbaef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.205925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a2b5f-2254-4936-a1a1-873e6fbaef86 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.206203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a2b5f-2254-4936-a1a1-873e6fbaef86 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.206474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a2b5f-2254-4936-a1a1-873e6fbaef86 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.207116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a2b5f-2254-4936-a1a1-873e6fbaef86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:35.207415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a2b5f-2254-4936-a1a1-873e6fbaef86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.208865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a2b5f-2254-4936-a1a1-873e6fbaef86 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.221359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 379/768] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:35.222591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753262b-e778-44b9-9af1-bbd8109449a1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.222876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753262b-e778-44b9-9af1-bbd8109449a1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.223214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753262b-e778-44b9-9af1-bbd8109449a1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.228660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 380/769] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:35.229473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753262b-e778-44b9-9af1-bbd8109449a1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.237197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463204a0-baec-4979-9c80-5750f8648ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.237810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463204a0-baec-4979-9c80-5750f8648ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.238616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463204a0-baec-4979-9c80-5750f8648ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.238901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463204a0-baec-4979-9c80-5750f8648ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.239560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463204a0-baec-4979-9c80-5750f8648ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:35.239897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463204a0-baec-4979-9c80-5750f8648ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.242162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463204a0-baec-4979-9c80-5750f8648ba6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.259011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 390/770] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:35.260539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421a23-b4b3-4cd6-ab56-eb6043d9baeb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.260823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421a23-b4b3-4cd6-ab56-eb6043d9baeb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.261260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421a23-b4b3-4cd6-ab56-eb6043d9baeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.279999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3a421a23-b4b3-4cd6-ab56-eb6043d9baeb None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:35.281176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a421a23-b4b3-4cd6-ab56-eb6043d9baeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:35.286853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:35.337756 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753262b-e778-44b9-9af1-bbd8109449a1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.358987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 391/771] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:35.359728 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421a23-b4b3-4cd6-ab56-eb6043d9baeb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.384652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.385324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.385654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.385928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.396066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.397584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:35.399064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:35.415851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:35.416936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:35.418022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:35.425319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 381/772] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:35.425319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.485393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421a23-b4b3-4cd6-ab56-eb6043d9baeb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.526140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.526763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.527042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.527690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] PATH_INFO: `/v3/services/network` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.528320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.529852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:35.541175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:35.542148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:35.543028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:35.547950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 392/773] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:35.548917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.562651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223bff0-2576-4e03-a89f-ec1631e5b914 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.581855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:35.619254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.620276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.620703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.621622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.630465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.631959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:35.643616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:35.644649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:35.645577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 382/774] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:35.646116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.672290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:35.680124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436b671-4bf2-4eb1-8a8e-84c78b317f21 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.719384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.720028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.720309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.720580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.721267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:35.721570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.722991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.736116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 393/775] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:35.736931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.772264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013dbf4-e588-44ef-bf0b-9f9acc413c4d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.810541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.811176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.811460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.811732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.812383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:35.812798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.814273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:35.823690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 383/776] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:35.823690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.852593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9eb801-3ff5-4e97-9ea7-4ecb4be63aed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.865431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:35.889692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.890323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.890605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.890868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.900997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:35.906792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:35.907823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 394/777] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:35.908674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:35.934159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:35.944457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f149623-d2a9-4f4d-a1cb-850fb6db1549 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:35.979427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:35.980081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:35.980357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:35.980630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:35.981316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:35.981622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:35.983615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.003546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 384/778] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:36.003546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.041092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2b145-5f21-4d29-afd9-09efe1d1f674 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.078746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.079354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.079677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.080130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.080960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=0b29d618b99e42aab0c42b61adf45e4f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:36.081266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.082693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.088660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 395/779] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/role_assignments?user.id=0b29d618b99e42aab0c42b61adf45e4f&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:36.089428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.131171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:36.138747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65310dd1-4b1a-4dca-b241-03bfccc00836 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.186797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:36.192802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.194310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.196032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.196535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.198046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] RBAC: Adding query filter params (type=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:36.198586 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.200733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.214276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 385/780] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:36.214276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.217197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe350a-d10f-4caf-b5e8-bf4e5b09a396 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.262930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.264895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.265801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.266437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.267396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:36.268160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.270459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.283173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 396/781] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:36.284438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.343211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450e61e-af89-4032-b936-b0af06761d27 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.378521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.379612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.380124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.380626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/6b6caf1ad3784ddabda79ca0b1fc468d/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.382745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=6b6caf1ad3784ddabda79ca0b1fc468d, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.384498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.402581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d770304-773e-4857-bf9b-742e38df8e85 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.405983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 386/782] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:36 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/6b6caf1ad3784ddabda79ca0b1fc468d/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:59:36.407098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.433685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:36.452355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.452899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.453187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.453466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.454357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] RBAC: Adding query filter params (interface=public, service_id=83c52771034a41529b0f762a821c1ab4, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:36.454669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.456747 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.467819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 397/783] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 00:59:36 2022] GET /identity/v3/endpoints?service_id=83c52771034a41529b0f762a821c1ab4&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:36.477852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.496834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:36.593792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b67a52-5408-44d1-aca9-e469bd69330b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.634731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.636024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.636564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.637028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.639659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23b2c-ce2b-4217-bd11-f6dca2b09456 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.649533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.651469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:36.672735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:36.673733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:36.674611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:36.674611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:36.674611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:36.676900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 387/784] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:36.687773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.691709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649b861-aaee-4698-883b-a15252933f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.692512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649b861-aaee-4698-883b-a15252933f46 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.692870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649b861-aaee-4698-883b-a15252933f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.693468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649b861-aaee-4698-883b-a15252933f46 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.693954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649b861-aaee-4698-883b-a15252933f46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.695351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649b861-aaee-4698-883b-a15252933f46 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.707137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 398/785] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 00:59:36 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:36.708544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e5e4-d7d0-4ab9-a0cb-c6ddd4a2d3fe None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.708826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e5e4-d7d0-4ab9-a0cb-c6ddd4a2d3fe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.709100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e5e4-d7d0-4ab9-a0cb-c6ddd4a2d3fe None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.710743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 399/786] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:36.716353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d3e03-7633-4749-bd16-59ffe38c05b5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.716650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d3e03-7633-4749-bd16-59ffe38c05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.716901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d3e03-7633-4749-bd16-59ffe38c05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.752649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-353d3e03-7633-4749-bd16-59ffe38c05b5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:36.753851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-353d3e03-7633-4749-bd16-59ffe38c05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:36.770583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:36.797695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741103-d789-4660-83a0-40357f1bd8e1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:36.841266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:36.841900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:36.842235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:36.842513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:36.843181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:36.843485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:36.847138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:36.869825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 388/787] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:36.880273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:36.904127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:36.911611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 400/788] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:37.001874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afbc23-cd03-49cf-b8d8-14487b0eb886 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:37.032182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:37.037211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:37.037847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:37.038120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:37.038388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:37.046856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:37.048321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:37.059975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:37.062814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:37.065242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 389/789] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:37.071772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d3e03-7633-4749-bd16-59ffe38c05b5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:37.219223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d3e03-7633-4749-bd16-59ffe38c05b5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:37.223581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:37.276941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:37.278032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:37.279306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:37.280887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:37.281823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:37.282495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:37.286068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:37.318769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 401/790] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 6 00:59:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:37.322977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:37.390522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23369e4-5aba-4547-a211-eaf97b07ced5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:37.427180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:37.458897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:37.459926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:37.460410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:37.460899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:37.461693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:37.464135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:37.480963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:37.481950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 00:59:37.483350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:37.491988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 390/791] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Sep 6 00:59:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:37.497574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:37.502857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f28ea-9912-44c7-8648-b04c95f479cc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:37.568713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:37.569683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:37.570231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:37.570699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:37.584029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:37.585840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:37.603716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:37.618004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:37.618961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:37.619902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:37.619902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:37.619902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:37.620550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 402/792] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:37.625267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:37.679098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a190d-13bb-419e-a0c2-2abf22c588c5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:37.738214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:37.738827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:37.739106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:37.739378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:37.740050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:37.740361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:37.741842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:37.747221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 391/793] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 6 00:59:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:37.752710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:37.856733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987411a-fbb0-49e9-a436-b5fbdef37550 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:37.870147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:37.897489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393e0a3-1ef5-4a09-aa6b-df5544edd154 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:37.917492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:37.918102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:37.918383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:37.918655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:37.919295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:37.919597 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:37.922421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:37.936630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:37.942274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 403/794] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:37.949811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:37.968999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:37.969617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:37.969901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:37.970174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:37.970805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:37.971105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:37.972501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:37.989854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 392/795] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 6 00:59:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:37.995312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:38.113555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3496508-7693-4867-84e6-93cfd002fa22 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:38.176688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:38.177372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.177653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.177924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.187889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:38.189546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:38.191697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47bfc6-5cc9-42b1-a502-f89d4a9e9eb2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:38.212970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:38.213909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:38.214807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:38.215680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 404/796] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:38.221152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:38.253783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e2b20-37a1-40d4-bcb1-ac41bc249d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:38.254401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2b20-37a1-40d4-bcb1-ac41bc249d52 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.254682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2b20-37a1-40d4-bcb1-ac41bc249d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.254967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2b20-37a1-40d4-bcb1-ac41bc249d52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.255627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e2b20-37a1-40d4-bcb1-ac41bc249d52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:38.257024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e2b20-37a1-40d4-bcb1-ac41bc249d52 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:38.273815 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:38.287657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 393/797] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 00:59:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:59:38.289163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a7140-a1d5-40ad-8de1-767b01b24253 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.289707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a7140-a1d5-40ad-8de1-767b01b24253 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.290436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a7140-a1d5-40ad-8de1-767b01b24253 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.292156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 394/798] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:38.298371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a58323-9281-49bf-9b7f-ffb1b946ed2e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.298992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a58323-9281-49bf-9b7f-ffb1b946ed2e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.299450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a58323-9281-49bf-9b7f-ffb1b946ed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.319274 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-40a58323-9281-49bf-9b7f-ffb1b946ed2e None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:38.320876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-40a58323-9281-49bf-9b7f-ffb1b946ed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:38.381306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:38.440661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6999a-1fef-4f01-99d7-81fff2b1410a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:38.463591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 395/799] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:38.495106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:38.495791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.496085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.496360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.497098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:38.497411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:38.499461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:38.512639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 405/800] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:38.518361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a58323-9281-49bf-9b7f-ffb1b946ed2e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:38.624903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a58323-9281-49bf-9b7f-ffb1b946ed2e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:38.661459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af5522-fd09-40d1-8aea-5b769060274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:38.662105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af5522-fd09-40d1-8aea-5b769060274d None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.662389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af5522-fd09-40d1-8aea-5b769060274d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.662661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af5522-fd09-40d1-8aea-5b769060274d None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/e26430820d684ca8b4f63e473a16b6ca/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.664617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af5522-fd09-40d1-8aea-5b769060274d None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=e26430820d684ca8b4f63e473a16b6ca, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:38.666315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af5522-fd09-40d1-8aea-5b769060274d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:38.684539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 396/801] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:38 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/e26430820d684ca8b4f63e473a16b6ca/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 00:59:38.785496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:38.802217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9771-f8b5-4907-b08f-41c464c16bf5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.802547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9771-f8b5-4907-b08f-41c464c16bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.802769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9771-f8b5-4907-b08f-41c464c16bf5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.804278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 397/802] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:38.810011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e78d-bbaf-42b0-8148-152a8f2c68ee None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.810310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e78d-bbaf-42b0-8148-152a8f2c68ee None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.810576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e78d-bbaf-42b0-8148-152a8f2c68ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.839173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f280e78d-bbaf-42b0-8148-152a8f2c68ee None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:38.840439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f280e78d-bbaf-42b0-8148-152a8f2c68ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:38.863525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:38.920107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:38.926196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 398/803] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:38.935420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d447a-e6ce-489b-90a0-2c471eaf3414 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:38.972675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486c0fb-34f1-4004-91ed-7470283a84a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:38.973732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486c0fb-34f1-4004-91ed-7470283a84a7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:38.974264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486c0fb-34f1-4004-91ed-7470283a84a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:38.974771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486c0fb-34f1-4004-91ed-7470283a84a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:38.986395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486c0fb-34f1-4004-91ed-7470283a84a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e486c0fb-34f1-4004-91ed-7470283a84a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:38.991547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:38.992491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:38.993324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:38.996009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 406/804] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:39.003767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e78d-bbaf-42b0-8148-152a8f2c68ee None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.129686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eda11b-6b78-4b06-8501-bd464b5c1c1a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.130323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eda11b-6b78-4b06-8501-bd464b5c1c1a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.130797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eda11b-6b78-4b06-8501-bd464b5c1c1a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.130901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e78d-bbaf-42b0-8148-152a8f2c68ee None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:39.132538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 407/805] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:39.140764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcd429-7064-4f80-92a8-23040d4381da None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.141059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcd429-7064-4f80-92a8-23040d4381da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.141312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcd429-7064-4f80-92a8-23040d4381da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.168799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:39.169414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.169696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.169968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.170660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:39.171209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:39.172365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:39.180944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 399/806] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:39.184798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b7bcd429-7064-4f80-92a8-23040d4381da None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:39.186033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b7bcd429-7064-4f80-92a8-23040d4381da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:39.190706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.325181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:39.332303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 408/807] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:39.334399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcd429-7064-4f80-92a8-23040d4381da None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.470410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:39.477794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697ed18-8acd-4a95-b9c8-3a478873796f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:39.507344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcd429-7064-4f80-92a8-23040d4381da None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:39.543524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:39.544175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.544449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.544714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.563548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:39.565024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:39.571844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:39.572710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.573235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.573715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.588910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:39.589810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:39.590713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:39.591642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:39.591642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:39.591642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:39.591642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 400/808] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:39.600594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:39.610170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:39.611073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:39.611960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:39.619386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 409/809] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:39.625067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.724155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:39.756276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07abf7f-044a-4b4f-81f6-df794cc97456 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:39.761441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84607e-cdb2-4326-b5b0-e94cf6b50b62 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:39.800941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:39.801889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.802119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.802441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.803616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:39.804172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:39.805075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:39.805694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.805980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.806245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.806346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:39.806896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:39.807197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:39.808597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:39.816283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 410/810] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:39.817188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.819809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:39.822291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 401/811] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:39.824019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.925510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba6b97-ab80-46e1-bae4-fdaa62f5c460 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:39.937613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3473a-9fce-4711-b646-0f43e0f11761 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:39.962430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:39.963418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.964039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.964516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] PATH_INFO: `/v3/services/network` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.965312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:39.966876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:39.969468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:39.976481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:39.977442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 00:59:39.978292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:39.979023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:39.979023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:39.979023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:39.979023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:39.981359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 411/812] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:39.982456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:39.989789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:39.991248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:40.008216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:40.009191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:40.010654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:40.010654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:40.010654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:40.010654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 402/813] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:40.010654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.054762 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:40.090147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128196d-61c3-4957-9114-bc944fe5774a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.120771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84739cc4-cbc4-45d1-b31b-a6faf645ea79 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.130652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.132606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.132903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.133183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.143280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.145132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:40.161206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:40.162138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 412/814] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 00:59:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:40.163670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.166563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.168263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.168458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.168693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.169340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:40.169640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.171579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.182371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 403/815] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:40.183511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.245319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:40.303337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35803266-59cd-40a1-b6ea-321c3178aa31 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.313874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c016-1405-4df1-92ce-ea8f849bc3ac None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.334966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:40.345642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.346391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.346777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.347156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.347941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:40.348348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.349811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.359099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.360062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.360488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.360906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.361781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:40.362249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.363774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 413/816] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:40.364087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.364904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.384007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 404/817] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:40.385300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.469362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663f29-ddc4-4bb5-b8a1-8209ea6bc410 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.495655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e5be8-99cf-45af-b563-95670700bca0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.498467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:40.506563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.507303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.508014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.508141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.508826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] RBAC: Adding query filter params (type=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:40.509199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.510750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.521678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 414/818] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:40.521678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.541823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.542871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.543333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.543837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.551849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.553629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:40.566083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:40.567020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 00:59:40.568067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:40.570705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 405/819] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:40.571732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:40.572209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.637293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1730c-32ac-462a-8ca2-fd5fbc5c591e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.674586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.675416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.675727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.676031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.684775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ddd1b-0aca-435a-9f2f-40cd02ab3d71 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.687663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.689149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:40.704736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:40.705695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:40.706583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:40.706583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:40.706583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:40.706583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 415/820] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:40.707248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.723660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.724494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.724780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.725053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.725768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.727359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.752326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 406/821] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 00:59:40 2022] POST /identity/v3/endpoints => generated 337 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:40.753166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:40.795581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:40.838703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ec4c6-b6c9-4a39-afaa-4599e7e93cfa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.840854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:40.888477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781ea83-ed2a-4115-ae49-6450310f1bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.889765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ea83-ed2a-4115-ae49-6450310f1bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.889942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af67b35-94b5-4108-880d-e16c05ad879a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:40.890062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ea83-ed2a-4115-ae49-6450310f1bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.890291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ea83-ed2a-4115-ae49-6450310f1bd0 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.891067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781ea83-ed2a-4115-ae49-6450310f1bd0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:40.891488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781ea83-ed2a-4115-ae49-6450310f1bd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.893269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781ea83-ed2a-4115-ae49-6450310f1bd0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.912453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 416/822] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:40.913514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a37145-617f-4e64-9eb1-1bfc5f362179 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.913797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a37145-617f-4e64-9eb1-1bfc5f362179 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.915941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a37145-617f-4e64-9eb1-1bfc5f362179 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.917353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 417/823] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:40.918456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eae01-b5b1-4919-b861-b73e43075061 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.918751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eae01-b5b1-4919-b861-b73e43075061 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.919034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eae01-b5b1-4919-b861-b73e43075061 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.920272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 418/824] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:40.923186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363dc57-e06c-451c-afc0-e0591b11cac4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.923487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363dc57-e06c-451c-afc0-e0591b11cac4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.924135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363dc57-e06c-451c-afc0-e0591b11cac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.931298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:40.932034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:40.932317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:40.932589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:40.933265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:40.933572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:40.934933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:40.946256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8363dc57-e06c-451c-afc0-e0591b11cac4 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:40.947825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8363dc57-e06c-451c-afc0-e0591b11cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:40.948886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 407/825] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:40.949832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.024763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 419/826] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:41.026353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed2b1-d297-4188-b603-60d4ab6339ff None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.026645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed2b1-d297-4188-b603-60d4ab6339ff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.026920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed2b1-d297-4188-b603-60d4ab6339ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.039226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.044545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-205ed2b1-d297-4188-b603-60d4ab6339ff None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:41.045992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-205ed2b1-d297-4188-b603-60d4ab6339ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:41.064846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3e55f-3a94-4dd6-af79-4028ad13e383 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.098045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.098675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.098954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.099225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.124182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 420/827] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:41.124182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed2b1-d297-4188-b603-60d4ab6339ff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.144425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.177677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.179198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.239443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed2b1-d297-4188-b603-60d4ab6339ff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.257473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.276105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.276953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.277538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.277951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.284898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.286503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.288437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 408/828] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 00:59:41.289513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:41.299084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:41.300182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:41.301255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:41.307490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 421/829] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:41.308472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.390244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.394889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38589181-aa72-4953-88f5-c6cc05d46a99 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.413358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3249b-3d4c-41bf-8881-9bdf9bfc2885 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.428330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.429034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.429328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.429581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.439838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.441274 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.446709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.447329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.447613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.447916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:41.455171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:41.456351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:41.457447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:41.457447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:41.457447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:41.457447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 409/830] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:41.457447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.460416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:41.465628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:41.466664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 422/831] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:41.467570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.488652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.557882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61158bee-f7e3-4896-8003-b88d95d5cf97 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.566614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15066873-9ee7-49d8-a4f2-7893f12366e8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.592837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.593439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.593715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.593983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.594659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:41.594959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.596410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.599964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.600584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.600861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.601134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.601701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.603037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.605555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.609270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 410/832] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:41.610073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:41.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:41.614475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:41.615380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:41.617597 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 423/833] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:41.618872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.696582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.708451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1266609-71a1-4f48-b11d-428f4d8856a7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.718178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04b43-fdc4-4596-8572-6a49963a3a87 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.742354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.742950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.743236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.743529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.744196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:41.744499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.745852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.751778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.752371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.752665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.752936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.753579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:41.753884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.755211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.756783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 411/834] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:41.757534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.767581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 424/835] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:41.768405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.830440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.863568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9dd06-a1ef-45f8-a3e7-f4fd12bfae40 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.869934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eec7ea-87bd-4253-b0ed-c1499f40926c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:41.896179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.896830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.897108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.897385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/0b29d618b99e42aab0c42b61adf45e4f/roles/b26b251c03cd4a2eac5f85927c38b797` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.901984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:41.902575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:41.902850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:41.903120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:41.903733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:41.904058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.905965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.910311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 425/836] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:41.911255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:41.911820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=0b29d618b99e42aab0c42b61adf45e4f, role_id=b26b251c03cd4a2eac5f85927c38b797)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:41.913239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:41.922402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:41.928138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 412/837] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:41 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/0b29d618b99e42aab0c42b61adf45e4f/roles/b26b251c03cd4a2eac5f85927c38b797 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:59:41.929091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.043111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c5607-68ef-4f58-b8fa-89e7dd77b408 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.044054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f5c83-ad2e-45bc-8978-f7843d4aefd3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.074615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:42.087045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.087217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.087618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.087723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.087965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.088064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.088233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.088335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.097928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.098962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.099657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.100421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:42.112739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:42.113642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 426/838] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:42.114920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:42.115465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:42.116359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:42.117959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:42.117959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:42.117959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:42.119308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 413/839] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:42.124312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.178553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:42.225949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ce200-83c0-4766-92dd-caee9424ba7b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.234557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef27b6-a375-4938-a63e-572b876bcfb9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.261579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.262191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.262473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.262745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.263367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:42.263668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.265646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.270477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 427/840] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:42.271263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.282413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.283351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.283849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.284312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.284977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:42.285286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.286785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.300377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 414/841] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:42.306347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.355285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:42.426991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9980813-9d17-4c2d-a9e4-0e5460f6ff4f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.428986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ca6c3-689e-465f-9afb-5ed772c73e7a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.441858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:42.473048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.473048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.473048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.473409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.477724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.478363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.478644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.478915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.479608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:42.479953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.480959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.481373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.482379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.494407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 428/842] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:42.497067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:42.498007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 00:59:42.499486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:42.501988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 415/843] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 00:59:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:42.502111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.507404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.612786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f479c-aeb0-47aa-8d51-07756f222588 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.624803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:42.626513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41180d-ab57-4c1d-acc0-9028de33ab32 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.650405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.651245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.651670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.652204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.653188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=6b6caf1ad3784ddabda79ca0b1fc468d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:42.653637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.655318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.663888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.664770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.665477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.665896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.666781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:42.667251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.668044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 429/844] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:42 2022] GET /identity/v3/role_assignments?user.id=6b6caf1ad3784ddabda79ca0b1fc468d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:42.669125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:42.692203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 416/845] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 6 00:59:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:42.698002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:42.818183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1245eb-e3ff-4de8-a129-8815363f87a6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:42.853699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:42.854307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:42.854583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:42.854845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:42.900846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:42.927372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:42.928854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:43.028614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 430/846] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 00:59:43.034970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:43.065960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:43.138396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42bd7c-e2ca-4e1d-a8e4-07882dae1c63 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:43.173957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:43.174599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:43.174860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:43.175134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:43.185147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:43.186626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:43.201118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:43.202168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:43.203072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:43.203072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:43.203072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:43.203072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 417/847] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:43.208404 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:43.285895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:43.314372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9213-fe40-4b85-aaf3-7384e05f475c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:43.349049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:43.349997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:43.350575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:43.351086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:43.351957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:43.352455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:43.354011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:43.369153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 431/848] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:43.381085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:43.470605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:43.492798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5b5d5-e0aa-43fe-8ac0-cd12ff97f628 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:43.527300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:43.527956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:43.528236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:43.528510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:43.535804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:43.537252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:43.548829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:43.549738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:43.550617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 418/849] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:43.557396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:43.682608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9932a62-3460-4bab-8e1c-0bf008bf5d7a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:43.718556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaaa36-9eee-4671-8ca4-e4357410c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:43.719168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaaa36-9eee-4671-8ca4-e4357410c161 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:43.719501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaaa36-9eee-4671-8ca4-e4357410c161 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:43.719797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaaa36-9eee-4671-8ca4-e4357410c161 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:43.720492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaaa36-9eee-4671-8ca4-e4357410c161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:43.720853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaaa36-9eee-4671-8ca4-e4357410c161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:43.722222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaaa36-9eee-4671-8ca4-e4357410c161 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:43.723816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:43.734654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 432/850] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:43.741078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:43.850304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7b377-9092-4ce4-b424-1d45b81a35e5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:43.885142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd97aa-00fa-4b72-ac4a-6fa6ec33347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:43.885964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd97aa-00fa-4b72-ac4a-6fa6ec33347a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:43.886247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd97aa-00fa-4b72-ac4a-6fa6ec33347a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:43.886516 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd97aa-00fa-4b72-ac4a-6fa6ec33347a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:43.887239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd97aa-00fa-4b72-ac4a-6fa6ec33347a None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=e26430820d684ca8b4f63e473a16b6ca) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:43.887602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd97aa-00fa-4b72-ac4a-6fa6ec33347a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:43.889593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd97aa-00fa-4b72-ac4a-6fa6ec33347a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:43.896135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 419/851] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:43 2022] GET /identity/v3/role_assignments?user.id=e26430820d684ca8b4f63e473a16b6ca&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:43.921537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7df318-a112-48f1-a389-028285c655ed None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:43.921864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7df318-a112-48f1-a389-028285c655ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:43.922099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7df318-a112-48f1-a389-028285c655ed None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:43.928048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 433/852] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:43.929624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb3e1b-f511-4c6b-bfc4-f35dd9852772 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:43.929912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb3e1b-f511-4c6b-bfc4-f35dd9852772 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:43.930191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb3e1b-f511-4c6b-bfc4-f35dd9852772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:43.938480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:43.951273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7ebb3e1b-f511-4c6b-bfc4-f35dd9852772 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:43.953068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7ebb3e1b-f511-4c6b-bfc4-f35dd9852772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:44.029422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 420/853] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:44.299604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7df318-a112-48f1-a389-028285c655ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:44.368536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d06b18-fe9e-452d-8e8d-3931a99ef88a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.369055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d06b18-fe9e-452d-8e8d-3931a99ef88a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.369303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d06b18-fe9e-452d-8e8d-3931a99ef88a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.370983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 421/854] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:44.376486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f450-5a1c-4bcd-9158-aa4acd86b3a5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.376876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f450-5a1c-4bcd-9158-aa4acd86b3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.377250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f450-5a1c-4bcd-9158-aa4acd86b3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.395383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b944f450-5a1c-4bcd-9158-aa4acd86b3a5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:44.396759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b944f450-5a1c-4bcd-9158-aa4acd86b3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:44.409312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7df318-a112-48f1-a389-028285c655ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:44.445581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:44.446222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.446504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.446786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.457849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:44.462685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:44.463636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:44.464513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 434/855] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 00:59:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:44.471078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:44.471981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 422/856] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:44.528087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:44.576620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27cb5e-4c88-46bc-9a80-7af09d1303c9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:44.612544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:44.613153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.613492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.613775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.614437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:44.614784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:44.616328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:44.625410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 435/857] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 00:59:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:44.630856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f450-5a1c-4bcd-9158-aa4acd86b3a5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:44.696114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:44.738194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f450-5a1c-4bcd-9158-aa4acd86b3a5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:44.754692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:44.772601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:44.773232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.773510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.773780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.786003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:44.788405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:44.808591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:44.809750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:44.810888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:44.810888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:44.810888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:44.813408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 423/858] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:44.823784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:44.874288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690de76-5bdc-405e-b794-b9a48833a417 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:44.891175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:44.913808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb54fc-b726-4890-97b4-60a48c0013ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:44.914435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb54fc-b726-4890-97b4-60a48c0013ef None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.914716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb54fc-b726-4890-97b4-60a48c0013ef None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.914989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb54fc-b726-4890-97b4-60a48c0013ef None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.915949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb54fc-b726-4890-97b4-60a48c0013ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:44.917307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb54fc-b726-4890-97b4-60a48c0013ef None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:44.924378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c8f1d-ab8f-46fa-b67d-0b202a602f75 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:44.933260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 436/859] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:59:44 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:59:44.934671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93eb90c-8647-4fba-9452-167caa0fc581 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.934956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93eb90c-8647-4fba-9452-167caa0fc581 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.935848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93eb90c-8647-4fba-9452-167caa0fc581 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.936519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 437/860] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:44.944937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5653e-40cc-4f2e-b023-36b71c817d0d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.945381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5653e-40cc-4f2e-b023-36b71c817d0d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.945787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5653e-40cc-4f2e-b023-36b71c817d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.947015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:44.975303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-66a5653e-40cc-4f2e-b023-36b71c817d0d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:44.977114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-66a5653e-40cc-4f2e-b023-36b71c817d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:44.995449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:44.996300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:44.996783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:44.997067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:44.998290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:44.998605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.000685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.019089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 424/861] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:45.024582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:45.083446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 438/862] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:45.147533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61fd4f-d162-4bde-a8e1-617cbc8ed85a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:45.182156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:45.182789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.183071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.183342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.190622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.192107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:45.205104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:45.206101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:45.207016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 425/863] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 00:59:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:45.211778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5653e-40cc-4f2e-b023-36b71c817d0d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:45.326825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5653e-40cc-4f2e-b023-36b71c817d0d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:45.352140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:45.356897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:45.361880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:45.364465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.364752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.365030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.365840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:45.366150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.367691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.387518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 439/864] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 6 00:59:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:45.392950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:45.394204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:45.473273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e42f7-9b1c-462f-b060-334c934bc920 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:45.508614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:45.509248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.509764 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.510259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.511041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.512732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.513165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70788c-3a4f-4b4a-a442-4c9a12eb6a25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:45.523027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:45.524027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:45.524903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:45.527779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 426/865] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Sep 6 00:59:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:45.533746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:45.563385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:45.564647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.565548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.566322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.610248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:45.644065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:45.651021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120687-5fdf-4840-a86a-4d48e9f2ba0b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:45.668059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.670513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.690333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:45.691172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.691555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.692035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.692821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:45.693223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.695031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.705088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 427/866] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:45.706920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:45.791875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 440/867] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:45 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 00:59:45.797649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:45.832558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc42a2-d6f3-4d4c-9098-4b170296aaaa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:45.871351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75901a93-c449-4ad1-a142-e8144411ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:45.872422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901a93-c449-4ad1-a142-e8144411ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.872909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901a93-c449-4ad1-a142-e8144411ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.873945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901a93-c449-4ad1-a142-e8144411ecd6 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.874930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75901a93-c449-4ad1-a142-e8144411ecd6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:45.875436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:45.876631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75901a93-c449-4ad1-a142-e8144411ecd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.878232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75901a93-c449-4ad1-a142-e8144411ecd6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.890279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 428/868] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:45.896401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f608-e4f5-457d-9dd0-533824f76238 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.897047 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f608-e4f5-457d-9dd0-533824f76238 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.897575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f608-e4f5-457d-9dd0-533824f76238 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.899513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 429/869] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:45.909315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ccfb-4503-4aa0-9063-9feb99f6bbff None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.909588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ccfb-4503-4aa0-9063-9feb99f6bbff None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.909860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3ccfb-4503-4aa0-9063-9feb99f6bbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.923016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a848074-c932-484c-83a1-84174d9bcacf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:45.930066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b7f3ccfb-4503-4aa0-9063-9feb99f6bbff None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:45.931275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b7f3ccfb-4503-4aa0-9063-9feb99f6bbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:45.952003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:45.964128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:45.965018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:45.965480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:45.965885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:45.977924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:45.979710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:45.996726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:45.997761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:45.999366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:45.999366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:45.999366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:46.001901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 441/870] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:46.007247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:46.020502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 430/871] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:46.121378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b8d96-382e-4d3c-969d-1907c35b350b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:46.138849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:46.159381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:46.160045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:46.160330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:46.160608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:46.161262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:46.161567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:46.162931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:46.176326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 442/872] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:46.181472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3ccfb-4503-4aa0-9063-9feb99f6bbff None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:46.282530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3ccfb-4503-4aa0-9063-9feb99f6bbff None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:46.290572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:46.318376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:46.319010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:46.319288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:46.319556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:46.328214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:46.328214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:46.340231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:46.341163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:46.342657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 431/873] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:46.350348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:46.372124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:46.437157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549dc3b-cc3e-4734-b6c9-7d02d3597987 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:46.461330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681723a1-3e86-4204-96e9-c174a5880598 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:46.482174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:46.483382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:46.483668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:46.483972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:46.484557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:46.486480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:46.498017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:46.498221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:46.499114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:46.500394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:46.502004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:46.502525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:46.503134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:46.504118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 443/874] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:46.504521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:46.504997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:46.505352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:46.507180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:46.521480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 432/875] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:46.526887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:46.554221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:46.631993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70558f7-24cf-4f5f-a6d8-7b292542bccc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:46.643185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:46.675599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b3077-55c7-45b8-b606-af6cb026e93e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:46.680427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:46.681035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:46.681316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:46.681588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:46.682215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:46.682516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:46.683911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:46.691840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 444/876] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:46.695899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:46.727016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f0cdf-acd6-4c32-9356-37628c4cab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:46.727781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f0cdf-acd6-4c32-9356-37628c4cab40 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:46.728066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f0cdf-acd6-4c32-9356-37628c4cab40 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:46.728340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f0cdf-acd6-4c32-9356-37628c4cab40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:46.729065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f0cdf-acd6-4c32-9356-37628c4cab40 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=0b29d618b99e42aab0c42b61adf45e4f) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:46.729630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f0cdf-acd6-4c32-9356-37628c4cab40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:46.731729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f0cdf-acd6-4c32-9356-37628c4cab40 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:46.739117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 433/877] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:46 2022] GET /identity/v3/role_assignments?user.id=0b29d618b99e42aab0c42b61adf45e4f&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:46.816017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c89a24-bccb-4a79-bf2f-7a62f98a3d4a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:46.820258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:46.869968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0762db5-bfa9-4aed-9bea-16c01f3f1a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:46.870956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762db5-bfa9-4aed-9bea-16c01f3f1a95 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:46.871445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762db5-bfa9-4aed-9bea-16c01f3f1a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:46.871979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0762db5-bfa9-4aed-9bea-16c01f3f1a95 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:46.872857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0762db5-bfa9-4aed-9bea-16c01f3f1a95 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:46.873413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0762db5-bfa9-4aed-9bea-16c01f3f1a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:46.875024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0762db5-bfa9-4aed-9bea-16c01f3f1a95 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:46.881024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 445/878] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:47.046514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb5b4-b0ff-4a0f-b70c-8566e4719288 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:47.046924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb5b4-b0ff-4a0f-b70c-8566e4719288 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:47.047104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb5b4-b0ff-4a0f-b70c-8566e4719288 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:47.048761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 434/879] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:47.056558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3183a-8dc3-47f1-bde4-c74d5f4c265d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:47.057021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3183a-8dc3-47f1-bde4-c74d5f4c265d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:47.057460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3183a-8dc3-47f1-bde4-c74d5f4c265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:47.079837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-acd3183a-8dc3-47f1-bde4-c74d5f4c265d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:47.081051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-acd3183a-8dc3-47f1-bde4-c74d5f4c265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:47.155215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 446/880] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:47.426174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb5b4-b0ff-4a0f-b70c-8566e4719288 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:47.535311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb5b4-b0ff-4a0f-b70c-8566e4719288 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:47.570824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:47.571450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:47.571725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:47.572022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:47.572600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:47.573962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:47.582948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:47.584004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:47.585049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 435/881] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:47.592841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3183a-8dc3-47f1-bde4-c74d5f4c265d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:47.697335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:47.701645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3183a-8dc3-47f1-bde4-c74d5f4c265d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:47.737068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:47.737785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:47.738068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:47.738378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:47.739010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:47.739312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:47.740691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:47.745539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 447/882] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:47.751234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:47.754683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:47.858723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130bbf5-02e9-4cd7-bb4e-242b18532c9a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:47.893408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf566f-3112-4753-bee5-adaa6302ed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:47.894011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf566f-3112-4753-bee5-adaa6302ed79 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:47.894286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf566f-3112-4753-bee5-adaa6302ed79 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:47.894560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf566f-3112-4753-bee5-adaa6302ed79 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:47.895185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf566f-3112-4753-bee5-adaa6302ed79 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:47.895483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf566f-3112-4753-bee5-adaa6302ed79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:47.896876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf566f-3112-4753-bee5-adaa6302ed79 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:47.902217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 436/883] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:47.909107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:47.960227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c449-912b-452d-8070-8dd45cf981b0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:47.960421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c449-912b-452d-8070-8dd45cf981b0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:47.960677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c449-912b-452d-8070-8dd45cf981b0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:47.962307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 437/884] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:47.968200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cf7b0-a535-45d6-8809-3c91724a3549 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:47.968709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cf7b0-a535-45d6-8809-3c91724a3549 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:47.969194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cf7b0-a535-45d6-8809-3c91724a3549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:47.987870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f25cf7b0-a535-45d6-8809-3c91724a3549 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:47.989401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f25cf7b0-a535-45d6-8809-3c91724a3549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:48.028810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009eeec-5665-4380-8cc9-973d56eea54b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:48.052800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:48.067370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 438/885] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:48.070896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:48.071515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:48.071823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:48.072105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:48.072813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] RBAC: Adding query filter params (interface=public, service_id=1093077c959648e29f30a9442ea1d12e, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:48.073124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:48.074482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:48.084839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 448/886] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 00:59:47 2022] GET /identity/v3/endpoints?service_id=1093077c959648e29f30a9442ea1d12e&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:48.086169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cf7b0-a535-45d6-8809-3c91724a3549 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:48.188482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cf7b0-a535-45d6-8809-3c91724a3549 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:48.235482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4b01c-178a-47f3-853e-b55ec9378ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:48.236906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4b01c-178a-47f3-853e-b55ec9378ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:48.237722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4b01c-178a-47f3-853e-b55ec9378ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:48.238447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4b01c-178a-47f3-853e-b55ec9378ca1 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:48.239606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4b01c-178a-47f3-853e-b55ec9378ca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:48.241269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4b01c-178a-47f3-853e-b55ec9378ca1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:48.247224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 439/887] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 00:59:48 2022] GET /identity/v3/services => generated 1011 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 00:59:48.424198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:48.466468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:48.559865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9521e1-0d88-4f89-91c1-e675e83f0ec2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:48.600031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630c691-bc8f-427d-89c7-8b26578f758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:48.601018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630c691-bc8f-427d-89c7-8b26578f758c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:48.601507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630c691-bc8f-427d-89c7-8b26578f758c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:48.601982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630c691-bc8f-427d-89c7-8b26578f758c None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:48.602868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630c691-bc8f-427d-89c7-8b26578f758c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:48.604490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630c691-bc8f-427d-89c7-8b26578f758c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:48.621410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 449/888] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:59:48 2022] POST /identity/v3/services => generated 269 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:48.838172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ba90-b336-4dc9-9a55-c7821b34ced2 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:48.838660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ba90-b336-4dc9-9a55-c7821b34ced2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:48.838903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ba90-b336-4dc9-9a55-c7821b34ced2 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:48.841348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 440/889] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:48.846955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e68f0-44f9-4572-a462-7da1059ec049 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:48.847371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e68f0-44f9-4572-a462-7da1059ec049 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:48.847550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e68f0-44f9-4572-a462-7da1059ec049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:48.866817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a9e68f0-44f9-4572-a462-7da1059ec049 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:48.868037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a9e68f0-44f9-4572-a462-7da1059ec049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:48.950211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 450/890] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:49.042137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6521-a1b9-4e92-bce6-0e5d82402319 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:49.042520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6521-a1b9-4e92-bce6-0e5d82402319 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:49.042890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6521-a1b9-4e92-bce6-0e5d82402319 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:49.045615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 441/891] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:49.051973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762b8ed-c5ac-4afc-9d41-1350d3fc501c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:49.052268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762b8ed-c5ac-4afc-9d41-1350d3fc501c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:49.052543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762b8ed-c5ac-4afc-9d41-1350d3fc501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:49.071447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a762b8ed-c5ac-4afc-9d41-1350d3fc501c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:49.072672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a762b8ed-c5ac-4afc-9d41-1350d3fc501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:49.080390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:49.152048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 451/892] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:49.241233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa6521-a1b9-4e92-bce6-0e5d82402319 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:49.318493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:49.367874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa6521-a1b9-4e92-bce6-0e5d82402319 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:49.409825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:49.410394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:49.410712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:49.410982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:49.422912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:49.427767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:49.428676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:49.429658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 442/893] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 6 00:59:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:49.434809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762b8ed-c5ac-4afc-9d41-1350d3fc501c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:49.459930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:49.567533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762b8ed-c5ac-4afc-9d41-1350d3fc501c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:49.580213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbc281-b584-480f-9c52-f0414af0e3ef None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:49.610220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:49.613982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:49.615870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:49.616648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:49.617176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:49.618229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:49.618683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:49.620943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:49.628584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:49.629586 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:49.630145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:49.630638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:49.631635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:49.631877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 452/894] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 6 00:59:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:49.633578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:49.641400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:49.648763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:49.669989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 443/895] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 00:59:49 2022] POST /identity/v3/services => generated 241 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:59:49.788454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c88c-5745-4594-a5c1-f4ff650fafdf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:49.838118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:49.838778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:49.839061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:49.839358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:49.852218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:49.853673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:49.869324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:49.870284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:49.871188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:49.871188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:49.871188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:49.871188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 453/896] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:49.876608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:49.982011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591044f-f072-4ff2-9ce1-8d2dcdf9950d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:50.018079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:50.019040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.019520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.020043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.020883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:50.021383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:50.022960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:50.035299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 444/897] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:50.040900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:50.098578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:50.152564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441f17a-2664-4124-924e-82ac4bedc6f6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:50.189815 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd21ed7-56e5-4d3a-94cd-6a63896716c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:50.190458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd21ed7-56e5-4d3a-94cd-6a63896716c2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.190798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd21ed7-56e5-4d3a-94cd-6a63896716c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.191076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd21ed7-56e5-4d3a-94cd-6a63896716c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.199987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd21ed7-56e5-4d3a-94cd-6a63896716c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:50.202145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd21ed7-56e5-4d3a-94cd-6a63896716c2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7dd21ed7-56e5-4d3a-94cd-6a63896716c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:50.214500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:50.215498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:50.216578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:50.220141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 454/898] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 00:59:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:50.221794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:50.272323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:50.299433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4eda4-d667-40a5-8aba-64103ec2f4d9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.299818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4eda4-d667-40a5-8aba-64103ec2f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.300030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4eda4-d667-40a5-8aba-64103ec2f4d9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.302689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 455/899] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:50.306782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704067e-9f53-493d-b582-544527291f4d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.307074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704067e-9f53-493d-b582-544527291f4d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.307350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704067e-9f53-493d-b582-544527291f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.327088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9704067e-9f53-493d-b582-544527291f4d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:50.328407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9704067e-9f53-493d-b582-544527291f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:50.339961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225d3f1-84aa-465a-8520-03367e8c76ae None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:50.384930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8334090-6e2c-4260-9313-1e6862d6e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:50.385540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8334090-6e2c-4260-9313-1e6862d6e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.385823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8334090-6e2c-4260-9313-1e6862d6e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.386095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8334090-6e2c-4260-9313-1e6862d6e9e1 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.386827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8334090-6e2c-4260-9313-1e6862d6e9e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:50.387131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8334090-6e2c-4260-9313-1e6862d6e9e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:50.389361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8334090-6e2c-4260-9313-1e6862d6e9e1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:50.407841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 456/900] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:50.415898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 445/901] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 6 00:59:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:50.422270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704067e-9f53-493d-b582-544527291f4d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:50.484179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:50.527182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704067e-9f53-493d-b582-544527291f4d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:50.563426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:50.564072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.564353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.564632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.645678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:50.647136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:50.669621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d1e53-19ed-408b-802d-3fa9af94e7f7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.670054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d1e53-19ed-408b-802d-3fa9af94e7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.670763 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d1e53-19ed-408b-802d-3fa9af94e7f7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.673112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 446/902] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:50.678573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50777912-ad30-4051-adf8-1205d087a80b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.678958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50777912-ad30-4051-adf8-1205d087a80b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.679133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50777912-ad30-4051-adf8-1205d087a80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.697314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-50777912-ad30-4051-adf8-1205d087a80b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:50.698482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50777912-ad30-4051-adf8-1205d087a80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:50.699482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:50.764612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 457/903] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 00:59:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 00:59:50.765697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:50.775627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 447/904] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:50.777393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50777912-ad30-4051-adf8-1205d087a80b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:50.818808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:50.876389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6abdcb-35a0-4607-aa57-0c77a85c363f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:50.893182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50777912-ad30-4051-adf8-1205d087a80b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:50.915799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:50.916659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.917092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.917609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.918425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:50.919953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:50.929711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:50.930412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:50.930758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:50.931095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:50.931328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:50.932543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 00:59:50.933592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:50.936152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 458/905] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:50.941488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:50.949284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:50.950879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:50.951946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:50.969954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:50.970971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:50.971920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:50.971920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:50.971920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:50.975929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 448/906] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:50.976759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.061809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:51.067383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05dacd-a55c-4d51-aa41-4e4774d92d06 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.086313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804e80-f237-4ffe-94b9-dc131cdcd00e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.110916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.111537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.111867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.112103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.112784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:51.113016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.114377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.119530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 459/907] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:51.120115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.130831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.131447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.131725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.132023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.132693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:51.132974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.134376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.148494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 449/908] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:51.149261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.200716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:51.235516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac7a8-6733-47a7-aeaf-b31b2273708f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.256776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d502db-e30c-44f2-8526-82aa6737fc56 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.273267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.274356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.274805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.275235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.275984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.278209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:51.288283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:51.289262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:51.290180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 460/909] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:51.291891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.296768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.297390 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.297671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.297944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.298572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:51.298938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.300761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.305980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 450/910] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:51.306743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.310988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:51.406965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed28f0-72f4-4e11-bb9f-2d4e3864c170 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.412533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829a34-bbd1-48ee-b5ac-3ca29387c69e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.448302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.448933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.449207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.449316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.449492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.449902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.450182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.450453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.451075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:51.451952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.452488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:51.453343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.458276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 451/911] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:51.458910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.459019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.460867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:51.474414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:51.475329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 461/912] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:51.476253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.533770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:51.570600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b20325-8f66-4d71-9908-41693c09ad4c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.595272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ad36-5571-47a3-bdff-87871fd05fc4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.610117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.610754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.611032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.611305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.611987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.613372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.632234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.632841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.633121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.633629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.633749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 452/913] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 00:59:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:51.634317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.640690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:51.641282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.642965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.656189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 462/914] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:51.659027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.752755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:51.800146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93a7f-22f8-4df2-9526-b9d196fb9920 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.808262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed467cee-4b0a-4d04-a6b7-8eabc7b26236 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:51.827786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:51.857376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.859039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.859584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.860049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.861016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:51.861302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:51.861705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.861909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:51.862187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:51.862456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:51.863153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] RBAC: Adding query filter params (interface=public, service_id=1e576cf6177f43749c396bf7404a0441, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:51.863458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:51.864295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.865528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:51.875776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 463/915] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/endpoints?service_id=1e576cf6177f43749c396bf7404a0441&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:51.877479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.882038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 453/916] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:51.887281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:51.991633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4f3cf-c0dc-4049-a5ee-e6c4e184541d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:52.004801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae6544-f771-4936-bc4b-eff3d41d5346 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:52.016871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:52.030159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510e1d7-87c0-4689-9325-25b6788cd31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:52.031123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e1d7-87c0-4689-9325-25b6788cd31d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:52.031123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e1d7-87c0-4689-9325-25b6788cd31d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:52.031295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e1d7-87c0-4689-9325-25b6788cd31d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:52.031993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510e1d7-87c0-4689-9325-25b6788cd31d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:52.033482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510e1d7-87c0-4689-9325-25b6788cd31d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:52.043816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 464/917] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 00:59:52.055107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734332-ec47-4832-a1c3-c57c23dfe24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:52.055736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734332-ec47-4832-a1c3-c57c23dfe24e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:52.056051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734332-ec47-4832-a1c3-c57c23dfe24e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:52.056321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734332-ec47-4832-a1c3-c57c23dfe24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:52.057055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734332-ec47-4832-a1c3-c57c23dfe24e None admin] RBAC: Adding query filter params (role.id=e412e312d7a0477abd23db933823bb46, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=0b29d618b99e42aab0c42b61adf45e4f) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:52.057357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734332-ec47-4832-a1c3-c57c23dfe24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:52.058857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734332-ec47-4832-a1c3-c57c23dfe24e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:52.071933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 454/918] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 00:59:51 2022] GET /identity/v3/role_assignments?user.id=0b29d618b99e42aab0c42b61adf45e4f&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=e412e312d7a0477abd23db933823bb46 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:52.219628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b22e37-2931-49c9-9493-1fd731da31a2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:52.220043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b22e37-2931-49c9-9493-1fd731da31a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:52.220213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b22e37-2931-49c9-9493-1fd731da31a2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:52.221817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 465/919] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:52.227071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2e0f-417a-42f7-8a2a-8e042c94037f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:52.227624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2e0f-417a-42f7-8a2a-8e042c94037f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:52.228001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2e0f-417a-42f7-8a2a-8e042c94037f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:52.249377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-960f2e0f-417a-42f7-8a2a-8e042c94037f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:52.250482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-960f2e0f-417a-42f7-8a2a-8e042c94037f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:52.292106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:52.320105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 455/920] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:52.585902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b22e37-2931-49c9-9493-1fd731da31a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:52.690944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b22e37-2931-49c9-9493-1fd731da31a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:52.726900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:52.727919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:52.728436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:52.728907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:52.729682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:52.731299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:52.740541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:52.741511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:52.742346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:52.745082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 466/921] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:52.751286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2e0f-417a-42f7-8a2a-8e042c94037f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:52.855595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2e0f-417a-42f7-8a2a-8e042c94037f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:52.889804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:52.890399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:52.890673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:52.890940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:52.891557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:52.891883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:52.893286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:52.898178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 456/922] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:52.903737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:52.956518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:53.010501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66931fb5-734e-429c-9644-a77c9861de2b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:53.045851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:53.046508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:53.046802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:53.047077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:53.047710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:53.048097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:53.049466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:53.054899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 467/923] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:53.061226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:53.101853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:53.166685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64631320-90d9-4e37-b627-f81e86343ae7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:53.203728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eff2f4-b1bf-4aaa-88ee-56558c0a9890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:53.204367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eff2f4-b1bf-4aaa-88ee-56558c0a9890 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:53.204642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eff2f4-b1bf-4aaa-88ee-56558c0a9890 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:53.204910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eff2f4-b1bf-4aaa-88ee-56558c0a9890 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:53.205612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eff2f4-b1bf-4aaa-88ee-56558c0a9890 None admin] RBAC: Adding query filter params (interface=public, service_id=24556f3b8e474b90be89e47a4c56bcc1, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:53.205919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eff2f4-b1bf-4aaa-88ee-56558c0a9890 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:53.207292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eff2f4-b1bf-4aaa-88ee-56558c0a9890 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:53.213154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 457/924] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 00:59:53 2022] GET /identity/v3/endpoints?service_id=24556f3b8e474b90be89e47a4c56bcc1&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:53.218816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:53.325531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef3507-994d-497d-bce6-821fdbf2f63a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:53.339289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:53.360557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ed729-02b1-486c-aff7-8b9c90ffce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:53.361166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ed729-02b1-486c-aff7-8b9c90ffce4e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:53.361449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ed729-02b1-486c-aff7-8b9c90ffce4e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:53.361724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ed729-02b1-486c-aff7-8b9c90ffce4e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:53.362327 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ed729-02b1-486c-aff7-8b9c90ffce4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:53.363745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ed729-02b1-486c-aff7-8b9c90ffce4e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:53.370140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 468/925] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 00:59:53 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 00:59:53.618155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99d32-c5b7-46b4-8cd2-15bbe3d0a616 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:53.618478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99d32-c5b7-46b4-8cd2-15bbe3d0a616 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:53.618713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99d32-c5b7-46b4-8cd2-15bbe3d0a616 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:53.620313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 458/926] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:53.626283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43cd7e-f4b9-4fe4-b2ec-def10aec21d7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:53.626571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43cd7e-f4b9-4fe4-b2ec-def10aec21d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:53.626851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43cd7e-f4b9-4fe4-b2ec-def10aec21d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:53.645779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3a43cd7e-f4b9-4fe4-b2ec-def10aec21d7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:53.646980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a43cd7e-f4b9-4fe4-b2ec-def10aec21d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:53.716342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 469/927] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:53.988756 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee99d32-c5b7-46b4-8cd2-15bbe3d0a616 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.064916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5873-6f94-4bf1-887b-9a3a29d49be2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.065435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5873-6f94-4bf1-887b-9a3a29d49be2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.065435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5873-6f94-4bf1-887b-9a3a29d49be2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.066773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 470/928] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:54.071959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf2ec5-cd0f-446a-9c6e-5fbb84f4c1fa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.072244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf2ec5-cd0f-446a-9c6e-5fbb84f4c1fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.072517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf2ec5-cd0f-446a-9c6e-5fbb84f4c1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.092704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4caf2ec5-cd0f-446a-9c6e-5fbb84f4c1fa None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:54.093982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4caf2ec5-cd0f-446a-9c6e-5fbb84f4c1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:54.094229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee99d32-c5b7-46b4-8cd2-15bbe3d0a616 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.105847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:54.134074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:54.134683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.134955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.135218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.135885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:54.137307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:54.140417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING keystone.server.flask.common [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 00:59:54.159261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.sql.core [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 00:59:54.159261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 00:59:54.159261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [parameters: {'id': '4724c6d301084a918170744790881de7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 00:59:54.159261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81610) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-84688420-85fa-40ca-bf69-b71459c234be None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4724c6d301084a918170744790881de7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4724c6d301084a918170744790881de7. Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application conn.query(q) Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 00:59:54.175737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application result.read() Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 00:59:54.176571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application next(self.gen) Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application yield resource Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application next(self.gen) Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise self.value Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application session.commit() Sep 06 00:59:54.177519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.session.flush() Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application compat.raise_( Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 00:59:54.178877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 00:59:54.179805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application conn.query(q) Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application result.read() Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 00:59:54.180598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [parameters: {'id': '4724c6d301084a918170744790881de7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.182839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:54.185024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4724c6d301084a918170744790881de7. Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 459/929] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Sep 6 00:59:53 2022] POST /identity/v3/projects => generated 262 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be30b8-5752-4a1a-9e53-ef7d9eb96e6d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.187206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be30b8-5752-4a1a-9e53-ef7d9eb96e6d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.188914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be30b8-5752-4a1a-9e53-ef7d9eb96e6d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.190527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 460/930] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:54.191163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be30b8-5752-4a1a-9e53-ef7d9eb96e6d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.191691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 471/931] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:54.192981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d6890-e280-4450-8063-46a5e10f3b61 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.193269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d6890-e280-4450-8063-46a5e10f3b61 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.193543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d6890-e280-4450-8063-46a5e10f3b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.218052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-514d6890-e280-4450-8063-46a5e10f3b61 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:54.219721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-514d6890-e280-4450-8063-46a5e10f3b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:54.260582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:54.314682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 472/932] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:54.320581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be30b8-5752-4a1a-9e53-ef7d9eb96e6d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.356432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:54.357363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.357854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.358317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.365360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:54.367211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:54.378718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:54.379677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:54.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.382369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 461/933] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:54.388431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d6890-e280-4450-8063-46a5e10f3b61 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.469208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.488916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:54.507043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d6890-e280-4450-8063-46a5e10f3b61 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.552282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:54.553552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.553986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.554808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.555995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:54.556452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:54.558753 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:54.577801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a147b-56d2-4ecb-9e96-5cbc35c76f07 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.583032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 473/934] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:54.608639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.629474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:54.630974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.631484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.631989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.632808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:54.635526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:54.641929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:54.652137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:54.653199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 00:59:54.654092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:54.657744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 462/935] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Sep 6 00:59:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:54.663342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.757120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2868a4-29ed-4cb1-a99d-fe9ee33111af None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.766092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:54.791707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c43db-3239-476f-9171-a0334bd72825 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.810066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:54.811533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.812043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.812509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.827111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:54.833664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:54.834600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:54.836068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 474/936] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 6 00:59:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:54.840537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.841737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:54.842600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.843083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.843575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.844450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:54.844977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:54.846581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:54.852528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 463/937] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:54.859649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:54.945191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:54.959559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914d14b-3e39-45b9-8554-2d00d4e1fe66 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.994007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ecc44-577c-4d61-8ff9-38f20253afa6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:54.996811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:54.997917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:54.997917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:54.998719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:54.999308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:54.999617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.001023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.010083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 475/938] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 6 00:59:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:55.015630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.036319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.036929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.037276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.037478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.038106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:55.038406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.039920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.051032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 464/939] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:55.052248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.066176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:55.150997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcece39-b556-4ec7-929e-72fe0e4ecdf4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:55.169329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472deca4-aeb1-4787-9557-ed81eff95aa7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:55.200898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb4f0-5b32-46c8-89be-500937f8be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.202081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb4f0-5b32-46c8-89be-500937f8be15 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.202525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb4f0-5b32-46c8-89be-500937f8be15 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.203334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb4f0-5b32-46c8-89be-500937f8be15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.208265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.209002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.209384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.209759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.210596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.212233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.216094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eb4f0-5b32-46c8-89be-500937f8be15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.218916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eb4f0-5b32-46c8-89be-500937f8be15 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7d2eb4f0-5b32-46c8-89be-500937f8be15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:55.236799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:55.237887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:55.238783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:55.238783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:55.238783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:55.245483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 465/941] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 00:59:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 00:59:55.245853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 476/941] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:55.246706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.300666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af00a0a-211f-4962-a5aa-6b4637ef1c96 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.301033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af00a0a-211f-4962-a5aa-6b4637ef1c96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.301225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af00a0a-211f-4962-a5aa-6b4637ef1c96 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.302881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 477/942] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:55.308503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148b9db-86cb-4c97-a81b-3440c6866c7c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.308874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148b9db-86cb-4c97-a81b-3440c6866c7c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.309042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148b9db-86cb-4c97-a81b-3440c6866c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.317682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:55.328668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:55.335063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3148b9db-86cb-4c97-a81b-3440c6866c7c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:55.336922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3148b9db-86cb-4c97-a81b-3440c6866c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:55.376557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab71fe-7538-4436-b61d-093c644bb09d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:55.420879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.421497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.421774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.422965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.422965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:55.422965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.424400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.439115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 478/944] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:55.441024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 466/944] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:55.444719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148b9db-86cb-4c97-a81b-3440c6866c7c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.551085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148b9db-86cb-4c97-a81b-3440c6866c7c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:55.582447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:55.587813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.588460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.589194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.589194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.597851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.599298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:55.611345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:55.612345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:55.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 479/945] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 00:59:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:55.620224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.708281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.742637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc84011-9c46-4056-80c0-076d9d619038 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:55.768685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:55.783320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.783969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.784254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.784530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.785267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:55.785578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.787015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.806061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 467/946] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 6 00:59:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:55.811598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.830228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a14895-4d6e-457e-92dc-46bd67711d6d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:55.873197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.874041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.874332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.874611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.875202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.876884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:55.889482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:55.890529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:55.891374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 480/947] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:55.896675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:55.936875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1e9f-3f7a-4b3a-87d7-a316a95da8a6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:55.970955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:55.972566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:55.973275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:55.973551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:55.973826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:55.984796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:55.986196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:56.002342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:56.003309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:56.004254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:56.004254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:56.004254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:56.006104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411543a9-584c-441e-a6cf-3634edc06670 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:56.008364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 468/948] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:56.010018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:56.042623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:56.043346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:56.044212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.044726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.045359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.046190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:56.046706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:56.048542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.053938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 481/949] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:56.062240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:56.115262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c993991-bbe2-405b-b2e8-37388c718258 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:56.150261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:56.150796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.151079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.151350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.152016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:56.152321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:56.153691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.166765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b13a6-5f3c-4e5f-a237-733e50d16977 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:56.170515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 469/950] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:56.171722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:56.200144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:56.205068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:56.205863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.206214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.206552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.207340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:56.207708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:56.209289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.219661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 482/951] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:56.220921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:56.268850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:56.278583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58aef8-7510-4673-a44e-d3839b7c5d9a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:56.317150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:56.317894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.318253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.318597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.326395 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:56.327786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.332798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d7a0d-922e-45d0-8ec2-c37ed1f000b5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 00:59:56.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:56.340559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 00:59:56.341616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:56.345647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 470/952] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:56.346848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:56.371552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:56.372209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.372619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.372897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.374233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:56.375595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.408483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 483/953] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 00:59:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 00:59:56.462543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c5053-73e9-4bd1-bf9d-c33b8c006868 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:56.512498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb829ce3-b55f-44b5-921e-660e4428d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:56.513563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb829ce3-b55f-44b5-921e-660e4428d67d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.514025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb829ce3-b55f-44b5-921e-660e4428d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.514435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb829ce3-b55f-44b5-921e-660e4428d67d None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.515300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb829ce3-b55f-44b5-921e-660e4428d67d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:56.515796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb829ce3-b55f-44b5-921e-660e4428d67d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:56.517408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb829ce3-b55f-44b5-921e-660e4428d67d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.531615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 471/954] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 00:59:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:56.541290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:56.600277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:56.662262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8518dc8-dc52-48b1-8a6e-4e89e842f7aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:56.695216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d6457-6e07-4ad9-a5e6-ad178d7674dc None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.695527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d6457-6e07-4ad9-a5e6-ad178d7674dc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.695809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d6457-6e07-4ad9-a5e6-ad178d7674dc None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.699921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 472/955] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 00:59:56.702784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02e70d-b0e8-4f7d-9de2-65746d0301d1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.703079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02e70d-b0e8-4f7d-9de2-65746d0301d1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.703194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:56.703434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02e70d-b0e8-4f7d-9de2-65746d0301d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.704345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:56.704847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:56.705314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/0b29d618b99e42aab0c42b61adf45e4f/roles/e412e312d7a0477abd23db933823bb46` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:56.707793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=0b29d618b99e42aab0c42b61adf45e4f, role_id=e412e312d7a0477abd23db933823bb46)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:56.709438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:56.725152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6d02e70d-b0e8-4f7d-9de2-65746d0301d1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:56.726900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d02e70d-b0e8-4f7d-9de2-65746d0301d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:56.727552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 484/956] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 00:59:56 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/0b29d618b99e42aab0c42b61adf45e4f/roles/e412e312d7a0477abd23db933823bb46 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 00:59:56.734435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:56.814642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 473/957] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:57.091186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:57.196517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:57.199564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f13c62-7fdf-4ac8-9cd9-d6131fc7295a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:57.235799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:57.236418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:57.236700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:57.236976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:57.237612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:57.239100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:57.241696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING keystone.server.flask.common [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 00:59:57.256194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 00:59:57.256194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 00:59:57.256194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [parameters: {'user_id': '5eba756d532747e0afd323226aff4d0a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 00:59:57.256194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-60749d37-371b-47dd-babb-46c9d68ba75a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 06 00:59:57.263074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 00:59:57.264540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 00:59:57.265528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 00:59:57.266500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 00:59:57.267437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 00:59:57.269797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'user_id': '5eba756d532747e0afd323226aff4d0a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:57.271059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 00:59:57.272421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 00:59:57.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 00:59:57.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 00:59:57.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 00:59:57.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 00:59:57.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 00:59:57.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 485/958] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 6 00:59:57 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:57.275498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02e70d-b0e8-4f7d-9de2-65746d0301d1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:57.371467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02e70d-b0e8-4f7d-9de2-65746d0301d1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:57.383270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6b824-8cd2-4987-8de5-382de2e3f33c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:57.383559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6b824-8cd2-4987-8de5-382de2e3f33c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:57.383854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6b824-8cd2-4987-8de5-382de2e3f33c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:57.393474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 486/959] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:57.394965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b11e-88c6-4ab2-91f4-8aca4ca85b26 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:57.395257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b11e-88c6-4ab2-91f4-8aca4ca85b26 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:57.395545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b11e-88c6-4ab2-91f4-8aca4ca85b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:57.406788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:57.407642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:57.408111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:57.408521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:57.417024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:57.417863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5417b11e-88c6-4ab2-91f4-8aca4ca85b26 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:57.419028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:57.419426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5417b11e-88c6-4ab2-91f4-8aca4ca85b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:57.424083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:57.433857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:57.435454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 00:59:57.437237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:57.444016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 474/960] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 00:59:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:57.445293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:57.519106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 487/961] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 00:59:57.570466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c5552-b14a-4a12-a6eb-2e09f734aa0c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:57.609065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:57.609674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:57.610007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:57.610284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:57.610980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:57.611325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:57.612852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:57.647807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 475/962] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 6 00:59:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:57.810763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417b11e-88c6-4ab2-91f4-8aca4ca85b26 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:57.918703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:57.929517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417b11e-88c6-4ab2-91f4-8aca4ca85b26 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:57.966238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:57.967416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:57.967709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:57.968008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:57.968582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:57.970067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:57.978845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:57.980134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 00:59:57.981444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 488/963] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 00:59:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:57.986157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:58.043638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:58.089314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850eb114-f255-44ba-a14b-2374ba56a66e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:58.123261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33a374-88d0-4bac-9822-3d0a3ea0176a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:58.123907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33a374-88d0-4bac-9822-3d0a3ea0176a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:58.124189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33a374-88d0-4bac-9822-3d0a3ea0176a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:58.124459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33a374-88d0-4bac-9822-3d0a3ea0176a None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:58.125091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33a374-88d0-4bac-9822-3d0a3ea0176a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:58.125393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33a374-88d0-4bac-9822-3d0a3ea0176a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:58.126760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33a374-88d0-4bac-9822-3d0a3ea0176a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:58.131685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 476/964] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:58.137096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:58.240515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb64f74-9706-403d-a3c9-7a4d20864486 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:58.275035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca0c71-835c-47ea-9ed2-be2ad2b48243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:58.275642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca0c71-835c-47ea-9ed2-be2ad2b48243 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:58.275948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca0c71-835c-47ea-9ed2-be2ad2b48243 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:58.276217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca0c71-835c-47ea-9ed2-be2ad2b48243 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:58.276844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca0c71-835c-47ea-9ed2-be2ad2b48243 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:58.277140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca0c71-835c-47ea-9ed2-be2ad2b48243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:58.278496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca0c71-835c-47ea-9ed2-be2ad2b48243 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:58.291783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 489/965] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 00:59:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:58.432027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49468-01d9-4d08-a226-7491a1ba337a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:58.432544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49468-01d9-4d08-a226-7491a1ba337a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:58.432697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49468-01d9-4d08-a226-7491a1ba337a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:58.434215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 477/966] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:58.439839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57069ac-9591-4385-8675-faca88044456 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:58.440126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57069ac-9591-4385-8675-faca88044456 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:58.440402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57069ac-9591-4385-8675-faca88044456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:58.459550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e57069ac-9591-4385-8675-faca88044456 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:58.460764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e57069ac-9591-4385-8675-faca88044456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:58.475588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:58.532042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 490/967] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:58.725337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a80cd6-bdf3-4ea8-8c5b-d5d915c42375 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:58.725681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a80cd6-bdf3-4ea8-8c5b-d5d915c42375 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:58.725896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a80cd6-bdf3-4ea8-8c5b-d5d915c42375 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:58.727368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 478/968] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 00:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 00:59:58.732403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc8cd-fb58-49a0-b3f7-28ab5c15f482 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:58.732699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc8cd-fb58-49a0-b3f7-28ab5c15f482 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:58.732971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc8cd-fb58-49a0-b3f7-28ab5c15f482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:58.751623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d0bdc8cd-fb58-49a0-b3f7-28ab5c15f482 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 00:59:58.752842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d0bdc8cd-fb58-49a0-b3f7-28ab5c15f482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 00:59:58.798843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a80cd6-bdf3-4ea8-8c5b-d5d915c42375 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:58.821971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 491/969] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 00:59:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 00:59:58.900565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a80cd6-bdf3-4ea8-8c5b-d5d915c42375 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:58.922001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:58.937682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:58.938306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:58.938581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:58.938848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:58.939421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:58.940811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:58.949491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:58.950486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 00:59:58.952456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:58.956143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 479/970] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 00:59:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:58.956979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdc8cd-fb58-49a0-b3f7-28ab5c15f482 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.036547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:59.056793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdc8cd-fb58-49a0-b3f7-28ab5c15f482 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.089001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.095028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.095640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.095954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.096226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.096852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:59.097148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.098524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.107951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 492/971] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:59.108911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.193669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f004165-3c71-43cf-bccc-ec1ec8eaf539 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.213003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c540ab3-bf48-4013-8182-a74d62022289 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.228060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.228689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.229023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.229304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.239259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 00:59:59.244475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:59.245396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:59.246234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 480/972] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:59.248839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.249446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.249877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.250149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.250997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:59.251106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.251839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.252431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.260199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 493/973] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:59.266211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.278328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:59.304391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:59.363622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96532fc-3241-4aa7-9994-eed8bbe0c344 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.371988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fe522-66e2-4af0-8e25-0fd0f1528591 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.402639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.403352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.403634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.403941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.404680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:59.404934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.406340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.408225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.408939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.409221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.409494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.410195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] RBAC: Adding query filter params (interface=admin, service_id=24556f3b8e474b90be89e47a4c56bcc1, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:59.410506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.411882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.414813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 481/974] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:59.421744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 494/975] 10.222.0.23 () {56 vars in 1249 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/endpoints?service_id=24556f3b8e474b90be89e47a4c56bcc1&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 00:59:59.422816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.422944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.493959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:59.526437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5e20-f746-487c-9906-124035923692 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.528029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418db3f-fec8-42e5-a3c5-34e5d43079ae None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.535499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:59.563355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.563984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.564261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.564538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.564853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.565141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.565480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.565761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.566027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.566574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.575038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 495/976] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 00:59:59.585691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.587339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 00:59:59.603962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:59.605191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 00:59:59.606358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 00:59:59.606358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 00:59:59.606358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:59.607973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 482/977] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 00:59:59.614422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.730464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e94f-97a1-4ffe-a1b9-c7f51b206b37 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.765652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.766518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.766952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.767328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.768125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 00:59:59.768533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.769992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.772673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 00:59:59.784568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 496/978] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 00:59:59.790094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 00:59:59.894598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b40c6-2605-4b39-8678-359b6b262adc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 00:59:59.928996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 00:59:59.929622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 00:59:59.929903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 00:59:59.930171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 00:59:59.938491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 00:59:59.939941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 00:59:59.950556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 00:59:59.951485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 00:59:59.957238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 483/979] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 00:59:59.959550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:00.064376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e2cef-1335-4476-add3-c27fabd6a728 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:00.099483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bbece-84e8-4158-8f78-f7d97585a06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:00.100119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bbece-84e8-4158-8f78-f7d97585a06d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.100402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bbece-84e8-4158-8f78-f7d97585a06d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.100677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bbece-84e8-4158-8f78-f7d97585a06d None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.101405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bbece-84e8-4158-8f78-f7d97585a06d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:00.101713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bbece-84e8-4158-8f78-f7d97585a06d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:00.103128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bbece-84e8-4158-8f78-f7d97585a06d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:00.124752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 497/980] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 6 00:59:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:00.125883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:00.145949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:00.220779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80459ca-5489-41e9-8250-5f21f674f621 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.221150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80459ca-5489-41e9-8250-5f21f674f621 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.221504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80459ca-5489-41e9-8250-5f21f674f621 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.223131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 498/981] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:00.228316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db3ba6-7a76-42e3-b4b3-3fa4184e2b70 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.228600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db3ba6-7a76-42e3-b4b3-3fa4184e2b70 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.228870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db3ba6-7a76-42e3-b4b3-3fa4184e2b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.232824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77aec0-02bb-465c-8a9a-c3680a012e71 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:00.250608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-37db3ba6-7a76-42e3-b4b3-3fa4184e2b70 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:00.251822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-37db3ba6-7a76-42e3-b4b3-3fa4184e2b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:00.269455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:00.270066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.270342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.270613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.304732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:00.322824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 499/982] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:00.340771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:00.342840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:00.360717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:00.448838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 484/983] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:00:00.450522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db3ba6-7a76-42e3-b4b3-3fa4184e2b70 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:00.550927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db3ba6-7a76-42e3-b4b3-3fa4184e2b70 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:00.591515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:00.592183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.592462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.592732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.596353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:00.604040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:00.605509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:00.620558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:00.626840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:00.627976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:00.627976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:00.627976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:00.627976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 500/984] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:00.627976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:00.666240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:00.714686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006fde2-4e02-46cc-96d7-20eb720d6303 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:00.732380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:00.736870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5197d-4420-48d4-8a16-a073563fecfd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:00.750809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:00.751417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.751695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.751996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.752658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:00.754023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:00.774025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 485/985] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 01:00:00 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:00:00.794446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:00.795571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.795884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.796158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.796822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:00.797125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:00.798537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:00.816679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 501/986] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:00.817963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:00.927997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e142a3f-6655-4f9f-852a-299ea38d9f7f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:00.967858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:00.968445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:00.968802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:00.969075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:00.975680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:00.977155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:00:00.988290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:00.989178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:00.990508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 486/987] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:00:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:00.995602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:01.089866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:01.100865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6cd4-6966-469e-9c26-34c6a05bf08c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:01.134439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9aef9-b58b-4afb-b827-2c32314eaedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:01.135422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9aef9-b58b-4afb-b827-2c32314eaedf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:01.135942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9aef9-b58b-4afb-b827-2c32314eaedf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:01.136437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9aef9-b58b-4afb-b827-2c32314eaedf None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:01.137324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9aef9-b58b-4afb-b827-2c32314eaedf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:01.137932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9aef9-b58b-4afb-b827-2c32314eaedf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:01.140114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9aef9-b58b-4afb-b827-2c32314eaedf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:01.152600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 502/988] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:00:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:01.158881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:01.200662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:01.264697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d5cf8-c7e5-4b9f-a2bd-948478c617a2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:01.297502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72120c-dbdd-4356-89c8-57f9d5de9861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:01.298105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72120c-dbdd-4356-89c8-57f9d5de9861 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:01.298385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72120c-dbdd-4356-89c8-57f9d5de9861 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:01.298657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72120c-dbdd-4356-89c8-57f9d5de9861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:01.299377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72120c-dbdd-4356-89c8-57f9d5de9861 None admin] RBAC: Adding query filter params (role.id=e412e312d7a0477abd23db933823bb46, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=0b29d618b99e42aab0c42b61adf45e4f) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:01.299684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72120c-dbdd-4356-89c8-57f9d5de9861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:01.301137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72120c-dbdd-4356-89c8-57f9d5de9861 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:01.307167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 487/989] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:01 2022] GET /identity/v3/role_assignments?user.id=0b29d618b99e42aab0c42b61adf45e4f&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=e412e312d7a0477abd23db933823bb46 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:01.524122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de13ae5-bffc-4ff0-8817-c9b1f9031223 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:01.524615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de13ae5-bffc-4ff0-8817-c9b1f9031223 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:01.524615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de13ae5-bffc-4ff0-8817-c9b1f9031223 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:01.526037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 503/990] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:01.531088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195d76-7ef8-40a1-9ca0-952c2172af04 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:01.531379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195d76-7ef8-40a1-9ca0-952c2172af04 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:01.531650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02195d76-7ef8-40a1-9ca0-952c2172af04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:01.550180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-02195d76-7ef8-40a1-9ca0-952c2172af04 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:01.551342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-02195d76-7ef8-40a1-9ca0-952c2172af04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:01.621948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 488/991] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:01.892087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de13ae5-bffc-4ff0-8817-c9b1f9031223 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:01.996581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de13ae5-bffc-4ff0-8817-c9b1f9031223 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:02.028932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:02.030067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:02.030345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:02.030615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:02.031184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:02.032563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:02.041384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:02.042372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 01:00:02.043212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:02.045451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 504/992] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 6 01:00:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:02.048876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195d76-7ef8-40a1-9ca0-952c2172af04 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:02.078642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:02.152214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02195d76-7ef8-40a1-9ca0-952c2172af04 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:02.187114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:02.187712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:02.188020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:02.188291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:02.188913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:02.189270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:02.190643 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:02.195527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 489/993] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 01:00:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:02.201174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:02.220018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:02.304350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccef98-77f7-467d-aa52-fa61fd283ef1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:02.337848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e3d3b-9e33-4278-9741-175f9f5bde59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:02.338455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3d3b-9e33-4278-9741-175f9f5bde59 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:02.338733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3d3b-9e33-4278-9741-175f9f5bde59 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:02.339003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3d3b-9e33-4278-9741-175f9f5bde59 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:02.339624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3d3b-9e33-4278-9741-175f9f5bde59 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:02.339955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3d3b-9e33-4278-9741-175f9f5bde59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:02.341301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3d3b-9e33-4278-9741-175f9f5bde59 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:02.346942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 505/994] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 6 01:00:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:02.352437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:02.452292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:02.458943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818d456-0366-4af3-89bc-7ac3695cb1ed None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:02.493118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894d1dd-5cf3-4f6f-b8cc-a29d165aee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:02.493726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894d1dd-5cf3-4f6f-b8cc-a29d165aee4e None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:02.494034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894d1dd-5cf3-4f6f-b8cc-a29d165aee4e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:02.494341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894d1dd-5cf3-4f6f-b8cc-a29d165aee4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:02.495019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894d1dd-5cf3-4f6f-b8cc-a29d165aee4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:02.496415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894d1dd-5cf3-4f6f-b8cc-a29d165aee4e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:02.519794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 490/995] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:00:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:00:02.716887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f780-10ce-4c05-b21a-11d9b5bb8e76 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:02.717379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f780-10ce-4c05-b21a-11d9b5bb8e76 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:02.717379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f780-10ce-4c05-b21a-11d9b5bb8e76 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:02.718854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 506/996] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:02.723962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c7ad-8841-47e7-9948-edd855185e7a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:02.724367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c7ad-8841-47e7-9948-edd855185e7a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:02.724548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04c7ad-8841-47e7-9948-edd855185e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:02.742407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ad04c7ad-8841-47e7-9948-edd855185e7a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:02.743601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad04c7ad-8841-47e7-9948-edd855185e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:02.819062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 491/997] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:03.094242 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3f780-10ce-4c05-b21a-11d9b5bb8e76 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:03.176848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:03.198566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3f780-10ce-4c05-b21a-11d9b5bb8e76 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:03.230820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a067fa-fcd9-48ae-ab66-704286ebcf1a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.231112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a067fa-fcd9-48ae-ab66-704286ebcf1a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.231386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a067fa-fcd9-48ae-ab66-704286ebcf1a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.232038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:03.233201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.233505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.233604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 492/998] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:03.233751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.234326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:03.235677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:03.238157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d7caa-8e75-41b7-af2b-de93d67e1121 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.238461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d7caa-8e75-41b7-af2b-de93d67e1121 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.238726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d7caa-8e75-41b7-af2b-de93d67e1121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:03.245183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:03.246187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:03.247032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 507/999] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:03.252425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:03.260515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-895d7caa-8e75-41b7-af2b-de93d67e1121 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:03.261784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-895d7caa-8e75-41b7-af2b-de93d67e1121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:03.327715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 493/1000] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:03.352245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b3c94-0d17-42c0-a2b1-3a06ea94d67b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:03.386108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:03.386710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.386988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.387259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.387905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:03.388206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:03.389591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:03.398954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 508/1001] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:03.400492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d7caa-8e75-41b7-af2b-de93d67e1121 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:03.416049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:03.506082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d7caa-8e75-41b7-af2b-de93d67e1121 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:03.542180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:03.542788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.543068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.543352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.543994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:03.544298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:03.546258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:03.556255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 494/1002] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:03.557109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:03.568499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:03.592941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:03.667393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3758fc-a81f-4d3e-9531-57df791b97ed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:03.668230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:03.691691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cefc2-66c2-4ea2-97fe-ae767fd16359 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:03.705510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:03.706112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.706391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.706671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.707362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] RBAC: Adding query filter params (interface=public, service_id=4a9dea2d1579434ea8ed27b4b669a859, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:03.707670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:03.709339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:03.714881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 509/1003] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/endpoints?service_id=4a9dea2d1579434ea8ed27b4b669a859&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:03.721168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:03.726275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:03.726964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.727245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.727538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] PATH_INFO: `/v3/services/image` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.728163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:03.729524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:03.738611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:03.739604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:03.744692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 495/1004] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:03.746833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:03.797394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:03.830469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019a222-e741-46d8-ae4e-9c4b246964bd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:03.846309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548b06f-639d-410f-b1e1-03df2733b8be None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:03.866254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:03.866865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.867163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.867429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.868058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:03.869655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:03.876968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 510/1005] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:00:03.883855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5e866-3560-43b4-b545-c7be5366d6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:03.884495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5e866-3560-43b4-b545-c7be5366d6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:03.884774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5e866-3560-43b4-b545-c7be5366d6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:03.885043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5e866-3560-43b4-b545-c7be5366d6c5 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:03.885710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5e866-3560-43b4-b545-c7be5366d6c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:03.886086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5e866-3560-43b4-b545-c7be5366d6c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:03.887521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5e866-3560-43b4-b545-c7be5366d6c5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:03.898652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 496/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:03.899674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:04.008297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482f01c-5efd-424c-a6b4-99a5fbd786bf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:04.046472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9c767-e9b8-4b42-be21-996a01858eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:04.047080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9c767-e9b8-4b42-be21-996a01858eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:04.047362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9c767-e9b8-4b42-be21-996a01858eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:04.047637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9c767-e9b8-4b42-be21-996a01858eb0 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:04.048318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9c767-e9b8-4b42-be21-996a01858eb0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:04.048644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9c767-e9b8-4b42-be21-996a01858eb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:04.049994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9c767-e9b8-4b42-be21-996a01858eb0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:04.052124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:04.056257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 511/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:04.428236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81897078-430d-4ced-a15a-b20ca8f99097 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:04.429124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81897078-430d-4ced-a15a-b20ca8f99097 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:04.429124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81897078-430d-4ced-a15a-b20ca8f99097 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:04.430377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 497/1008] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:04.440277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718d252-2b40-4a90-8483-0994a80b1781 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:04.440568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718d252-2b40-4a90-8483-0994a80b1781 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:04.440838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718d252-2b40-4a90-8483-0994a80b1781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:04.460488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3718d252-2b40-4a90-8483-0994a80b1781 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:04.461680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3718d252-2b40-4a90-8483-0994a80b1781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:04.535398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 512/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:04.803063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81897078-430d-4ced-a15a-b20ca8f99097 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:04.807146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:04.918369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81897078-430d-4ced-a15a-b20ca8f99097 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:04.953631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fd218-5bc7-44fc-b637-3875aa951416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:04.954288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd218-5bc7-44fc-b637-3875aa951416 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:04.954571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd218-5bc7-44fc-b637-3875aa951416 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:04.954844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd218-5bc7-44fc-b637-3875aa951416 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:04.956012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fd218-5bc7-44fc-b637-3875aa951416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:04.957460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fd218-5bc7-44fc-b637-3875aa951416 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:04.963855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 498/1010] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:04.965306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718d252-2b40-4a90-8483-0994a80b1781 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:05.067106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718d252-2b40-4a90-8483-0994a80b1781 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:05.095204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:05.101967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3471d6-de40-4687-a736-ee114d6960fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:05.102569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3471d6-de40-4687-a736-ee114d6960fd None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:05.102848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3471d6-de40-4687-a736-ee114d6960fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:05.103121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3471d6-de40-4687-a736-ee114d6960fd None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:05.103793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3471d6-de40-4687-a736-ee114d6960fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:05.105206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3471d6-de40-4687-a736-ee114d6960fd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:05.137329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 513/1011] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 01:00:04 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:05.781757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f68db-5914-4e78-a18f-0cf236452467 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:05.782092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f68db-5914-4e78-a18f-0cf236452467 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:05.782316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f68db-5914-4e78-a18f-0cf236452467 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:05.783847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 499/1012] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:05.788980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ecd2c-2611-4d41-9a13-6b26fdfa67cf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:05.789279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ecd2c-2611-4d41-9a13-6b26fdfa67cf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:05.789555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ecd2c-2611-4d41-9a13-6b26fdfa67cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:05.807777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5b2ecd2c-2611-4d41-9a13-6b26fdfa67cf None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:05.808977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5b2ecd2c-2611-4d41-9a13-6b26fdfa67cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:05.879859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 514/1013] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:05.962768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c2b90-5c63-43ad-954b-4fd5c0e4427d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:05.963270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c2b90-5c63-43ad-954b-4fd5c0e4427d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:05.963347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c2b90-5c63-43ad-954b-4fd5c0e4427d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:05.964952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 500/1014] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:05.969913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc51615-fa77-4fcd-a0ab-e5e1d3bde806 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:05.970316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc51615-fa77-4fcd-a0ab-e5e1d3bde806 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:05.970547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc51615-fa77-4fcd-a0ab-e5e1d3bde806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:05.988691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1dc51615-fa77-4fcd-a0ab-e5e1d3bde806 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:05.990015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1dc51615-fa77-4fcd-a0ab-e5e1d3bde806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:06.059083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 515/1015] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:06.148093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c2b90-5c63-43ad-954b-4fd5c0e4427d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:06.220112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:06.254370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c2b90-5c63-43ad-954b-4fd5c0e4427d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:06.287469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:06.288119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:06.288400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:06.288672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:06.289246 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:06.290851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:06.299944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:06.300904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:06.304774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 501/1016] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:00:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:06.307175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc51615-fa77-4fcd-a0ab-e5e1d3bde806 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:06.330417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:06.354519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:06.416983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc51615-fa77-4fcd-a0ab-e5e1d3bde806 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:06.438597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba172c7-423b-4c68-9412-69c159a804c1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:06.453114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:06.453723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:06.454005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:06.454279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:06.454904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:06.455205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:06.456587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:06.461366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 516/1017] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 01:00:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:06.467516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:06.477394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:06.478002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:06.478282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:06.478556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:06.479199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:06.480595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:06.485418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:06.507808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 502/1018] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 01:00:06 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:00:06.596602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b77193-41de-487b-908b-b790feaa6b79 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:06.604368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:06.642376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b504-dd94-4a04-a885-17067b1de461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:06.642988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b504-dd94-4a04-a885-17067b1de461 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:06.643271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b504-dd94-4a04-a885-17067b1de461 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:06.643547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b504-dd94-4a04-a885-17067b1de461 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:06.644209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b504-dd94-4a04-a885-17067b1de461 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:06.644515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b504-dd94-4a04-a885-17067b1de461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:06.645887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b504-dd94-4a04-a885-17067b1de461 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:06.651703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 517/1019] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 6 01:00:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:06.657997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:06.761416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf9b6c-da2c-4bad-bd09-1ac27905d8d0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:06.795905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3552a6-eed3-41d5-af4a-00daa6ae5353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:06.796520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3552a6-eed3-41d5-af4a-00daa6ae5353 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:06.796804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3552a6-eed3-41d5-af4a-00daa6ae5353 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:06.797076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3552a6-eed3-41d5-af4a-00daa6ae5353 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:06.797750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3552a6-eed3-41d5-af4a-00daa6ae5353 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:06.799124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3552a6-eed3-41d5-af4a-00daa6ae5353 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:06.818521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 503/1020] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:00:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:07.106499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c7f7-3169-4ac9-8b64-73b859236c52 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:07.106990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c7f7-3169-4ac9-8b64-73b859236c52 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:07.106990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c7f7-3169-4ac9-8b64-73b859236c52 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:07.108661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 518/1021] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:07.113989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746143ff-ca7b-4c97-97cd-c6320a878655 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:07.114403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746143ff-ca7b-4c97-97cd-c6320a878655 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:07.114598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746143ff-ca7b-4c97-97cd-c6320a878655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:07.132652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-746143ff-ca7b-4c97-97cd-c6320a878655 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:07.133820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-746143ff-ca7b-4c97-97cd-c6320a878655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:07.184842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:07.214134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 504/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:07.478690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c7f7-3169-4ac9-8b64-73b859236c52 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:07.587628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c7f7-3169-4ac9-8b64-73b859236c52 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:07.624321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f573a4-4f79-4bdb-9c37-7c0f30909b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:07.624971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f573a4-4f79-4bdb-9c37-7c0f30909b5e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:07.625236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f573a4-4f79-4bdb-9c37-7c0f30909b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:07.625509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f573a4-4f79-4bdb-9c37-7c0f30909b5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:07.626623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f573a4-4f79-4bdb-9c37-7c0f30909b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:07.628072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f573a4-4f79-4bdb-9c37-7c0f30909b5e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:07.631788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 519/1023] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:07.635222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746143ff-ca7b-4c97-97cd-c6320a878655 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:07.735056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746143ff-ca7b-4c97-97cd-c6320a878655 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:07.768101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236169a0-e354-4c2e-a140-dfb2e8932e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:07.768712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236169a0-e354-4c2e-a140-dfb2e8932e65 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:07.768989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236169a0-e354-4c2e-a140-dfb2e8932e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:07.769259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236169a0-e354-4c2e-a140-dfb2e8932e65 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:07.769898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236169a0-e354-4c2e-a140-dfb2e8932e65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:07.771310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236169a0-e354-4c2e-a140-dfb2e8932e65 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:07.801277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 505/1024] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 01:00:07 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:08.462817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cac95-7d9e-46d9-b85a-78dbfc5c7f48 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:08.463358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cac95-7d9e-46d9-b85a-78dbfc5c7f48 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:08.463358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cac95-7d9e-46d9-b85a-78dbfc5c7f48 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:08.464783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 520/1025] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:08.470532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91771c-c512-4ba8-a377-a831da260175 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:08.470938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91771c-c512-4ba8-a377-a831da260175 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:08.471104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91771c-c512-4ba8-a377-a831da260175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:08.489434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7b91771c-c512-4ba8-a377-a831da260175 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:08.490624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7b91771c-c512-4ba8-a377-a831da260175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:08.517368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:08.559967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 506/1026] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:08.819696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f7a70-1e7f-4bbd-8e7f-e5ebd77278a9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:08.820069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f7a70-1e7f-4bbd-8e7f-e5ebd77278a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:08.820235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f7a70-1e7f-4bbd-8e7f-e5ebd77278a9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:08.821665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 521/1027] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:08.827346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cb3e3-fe92-4327-98c3-a3dda4fd9c84 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:08.827635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cb3e3-fe92-4327-98c3-a3dda4fd9c84 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:08.827941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cb3e3-fe92-4327-98c3-a3dda4fd9c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:08.831284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f7a70-1e7f-4bbd-8e7f-e5ebd77278a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:08.849988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8a4cb3e3-fe92-4327-98c3-a3dda4fd9c84 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:08.851453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8a4cb3e3-fe92-4327-98c3-a3dda4fd9c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:08.869870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:08.923455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 507/1028] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:08.948340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f7a70-1e7f-4bbd-8e7f-e5ebd77278a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:08.982867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:08.983494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:08.983798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:08.984078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:08.984648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:08.986001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:08.995202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:08.996215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:09.000844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 522/1029] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Sep 6 01:00:08 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:09.002950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cb3e3-fe92-4327-98c3-a3dda4fd9c84 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.095781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:09.103239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cb3e3-fe92-4327-98c3-a3dda4fd9c84 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.136973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.137575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.137858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.138133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.138782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] RBAC: Adding query filter params (name=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:09.139091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:09.140493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:09.149904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 508/1030] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:09.150856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.189933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.250519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:09.257114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e5dd4-a73c-43e2-bf01-8b363a209597 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.292598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.293207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.293489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.293764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.294394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:09.295241 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:09.296743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:09.301699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 523/1031] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:09.307430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.314169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd9ad-8593-49a6-8edd-f17001ff326e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.351233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.351936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.352194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.352463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.363303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:09.368164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:09.369099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:09.369938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 509/1032] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:09.375574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.395465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:09.418712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5a63c-f85d-4537-8a99-a56bf479a00e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.456249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.456921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.457478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.457478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.458197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] RBAC: Adding query filter params (interface=public, service_id=b2cab010316e4167a4247ab0c01e77c8, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:09.458507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:09.459896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:09.464996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 524/1033] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/endpoints?service_id=b2cab010316e4167a4247ab0c01e77c8&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:09.470850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.487762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021478e-abb0-45d5-8abe-a367c65f0597 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.500352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:09.524716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.525315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.525589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.525857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.526512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:09.526809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:09.528207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:09.537265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 510/1034] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:09.542944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.581573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8b34f-9065-40e0-a71f-2cfda01250c1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.617389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.617963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.618260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.618569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.619324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:09.621239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:09.636445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 525/1035] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:00:09.636653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:09.657937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5e63e-56d3-495d-ae78-deedee1c375a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.698173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835804f-300d-470e-a20f-72e8cc1fc927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.698828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835804f-300d-470e-a20f-72e8cc1fc927 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.699150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835804f-300d-470e-a20f-72e8cc1fc927 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.699427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835804f-300d-470e-a20f-72e8cc1fc927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.711849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835804f-300d-470e-a20f-72e8cc1fc927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:09.713324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835804f-300d-470e-a20f-72e8cc1fc927 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5835804f-300d-470e-a20f-72e8cc1fc927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:09.729517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:09.730562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:09.737214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:09.737214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:09.737214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:09.737214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 511/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:09.738978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.770080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62978dd-b538-44d5-b3bd-02b4e7e361ea None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.770403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62978dd-b538-44d5-b3bd-02b4e7e361ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.770646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62978dd-b538-44d5-b3bd-02b4e7e361ea None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.772186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 512/1037] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 01:00:09.776968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744c06-c3e4-48f3-ae47-e9b9bc0992ff None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.777258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744c06-c3e4-48f3-ae47-e9b9bc0992ff None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.777534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744c06-c3e4-48f3-ae47-e9b9bc0992ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.796593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fe744c06-c3e4-48f3-ae47-e9b9bc0992ff None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:09.797783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fe744c06-c3e4-48f3-ae47-e9b9bc0992ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:09.841008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:09.853226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d166d-fbf0-4869-8e15-b7e6469f228a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:09.874655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 513/1038] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:09.889792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:09.890399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:09.890677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:09.890943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:09.891575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:09.891903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:09.893259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:09.911777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 526/1039] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:09.913864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe744c06-c3e4-48f3-ae47-e9b9bc0992ff None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:09.977070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.018399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe744c06-c3e4-48f3-ae47-e9b9bc0992ff None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.052808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.053429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.053713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.053979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.061096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.062524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:00:10.074603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:10.075571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:10.080921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 514/1040] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 01:00:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:10.084283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.147227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.188959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a139c-7ca2-4e41-b25e-52dd86b0ceed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.225216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.225814 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.226095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.226368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.227085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:10.227393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.228843 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.252455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 527/1041] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:10.253563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.261305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6120bc7-3e4c-417b-8bd7-d39e3fae2e19 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.264634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.286322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.300815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.301594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.301872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.302142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.314190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:10.319335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:10.320343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:10.321182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 515/1042] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:10.326933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.362188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9852ce-b37e-4f32-882d-d1cf9b7e9521 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.399178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.399700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.400018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.400287 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.433961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796430b-7d97-40ef-8dc4-ef503041a144 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.468257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.468858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.469139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.469409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.470056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:10.470354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.471715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.475423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.477018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.481521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 516/1043] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:10.486776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.488135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.536855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.596704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197f3a6-12e8-4d14-a5cc-780425464ded None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.604302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 528/1044] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 01:00:10.610104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.636723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.637352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.637634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.637902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] PATH_INFO: `/v3/users/2fbc15c528c84b789758516262935bbd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.652254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] RBAC: Authorizing `identity:get_user(user_id=2fbc15c528c84b789758516262935bbd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.653878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.655709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 517/1045] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/users/2fbc15c528c84b789758516262935bbd => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:10.661312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.724433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f90db-d808-492d-82c4-5efac1755528 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.735354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.763400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.764044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.764329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.764733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.767605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b536e3-76fc-40ca-a900-54d6ac342031 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.769353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.774972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.776448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:10.793412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:10.794406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:10.795845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:10.795845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:10.795845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:10.795845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 529/1046] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:10.800561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.806901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.807586 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.807957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.808301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.880295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.881742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.905917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08d02d-2769-4b80-a75b-2f75e1090687 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:10.942503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:10.943110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:10.943391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:10.943664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:10.944335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:10.944636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:10.946113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:10.958756 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 530/1047] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:10.963989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.986312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:10.987974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 518/1048] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 01:00:10.993143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:10.996109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:11.074173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf65e-53f9-4ff1-8d41-ec383264666c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:11.097947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73042741-2437-4565-9fbb-2390586a5a37 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:11.113922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:11.114717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:11.114998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:11.115276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:11.122122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:11.123596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:11.133181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:11.133810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:11.134088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:11.134358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:00:11.134869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:11.135708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:11.136627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:11.137061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] RBAC: Authorizing `identity:get_project(project_id=7868e14f5def455999e86ee4ba979939)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:11.137061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:11.143589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 519/1049] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/projects/7868e14f5def455999e86ee4ba979939 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:11.143722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 531/1050] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:11.143827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:11.144499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:11.213106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:11.229603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:11.246121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6aa5d-fb8a-497b-a3e6-6f51358e3d61 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:11.254079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c1b2a-ba1f-424d-9d34-e0fe61906a85 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:11.282846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:11.283510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:11.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:11.284310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:11.285107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:11.285531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:11.286939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:11.290134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186f9ea-e245-494b-b260-f024145f5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:11.290984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f9ea-e245-494b-b260-f024145f5d52 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:11.291268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f9ea-e245-494b-b260-f024145f5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:11.291540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f9ea-e245-494b-b260-f024145f5d52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:11.292290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f9ea-e245-494b-b260-f024145f5d52 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=7868e14f5def455999e86ee4ba979939, user.id=2fbc15c528c84b789758516262935bbd) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:11.292601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f9ea-e245-494b-b260-f024145f5d52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:11.294011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f9ea-e245-494b-b260-f024145f5d52 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:11.299407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 520/1051] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:11.305761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:11.308180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 532/1052] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:11 2022] GET /identity/v3/role_assignments?user.id=2fbc15c528c84b789758516262935bbd&scope.project.id=7868e14f5def455999e86ee4ba979939&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:11.416596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93ec2f-27f4-4bb7-a34e-1755dda3f676 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:11.447362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:11.454587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae59d7-53a6-408c-b4ff-d469490da476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:11.455455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae59d7-53a6-408c-b4ff-d469490da476 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:11.455970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae59d7-53a6-408c-b4ff-d469490da476 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:11.456445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae59d7-53a6-408c-b4ff-d469490da476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:11.457394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae59d7-53a6-408c-b4ff-d469490da476 None admin] RBAC: Adding query filter params (role.id=e412e312d7a0477abd23db933823bb46, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=6b6caf1ad3784ddabda79ca0b1fc468d) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:11.457898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae59d7-53a6-408c-b4ff-d469490da476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:11.459538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae59d7-53a6-408c-b4ff-d469490da476 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:11.465771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 521/1053] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:11 2022] GET /identity/v3/role_assignments?user.id=6b6caf1ad3784ddabda79ca0b1fc468d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=e412e312d7a0477abd23db933823bb46 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:11.567892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e162a1-ecc5-4ecf-af0d-78781ee8a649 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:11.568440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e162a1-ecc5-4ecf-af0d-78781ee8a649 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:11.568528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e162a1-ecc5-4ecf-af0d-78781ee8a649 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:11.570052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 533/1054] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:11.575638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724e5e4-07ba-4eb8-bffc-98ba395e02f8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:11.576646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724e5e4-07ba-4eb8-bffc-98ba395e02f8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:11.576751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724e5e4-07ba-4eb8-bffc-98ba395e02f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:11.597067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c724e5e4-07ba-4eb8-bffc-98ba395e02f8 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:11.598267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c724e5e4-07ba-4eb8-bffc-98ba395e02f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:11.667795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 522/1055] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:11.938142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e162a1-ecc5-4ecf-af0d-78781ee8a649 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:12.039969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e162a1-ecc5-4ecf-af0d-78781ee8a649 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:12.073986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:12.074612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:12.074893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:12.075167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:12.075739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:12.077137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:12.085848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:12.086832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 01:00:12.087666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:12.091923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 534/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Sep 6 01:00:11 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:12.093145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724e5e4-07ba-4eb8-bffc-98ba395e02f8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:12.192414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724e5e4-07ba-4eb8-bffc-98ba395e02f8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:12.215309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:12.226750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:12.227352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:12.227628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:12.227926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:12.228549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] RBAC: Adding query filter params (name=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:12.228843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:12.230207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:12.239778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 523/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:12.242380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:12.255263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:12.347678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab79826-9ae6-4ed0-acd8-f9f3ef04c827 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:12.385257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693396-0ed8-41a1-a068-a02037ab6067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:12.385874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693396-0ed8-41a1-a068-a02037ab6067 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:12.386156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693396-0ed8-41a1-a068-a02037ab6067 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:12.386430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693396-0ed8-41a1-a068-a02037ab6067 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:12.387050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693396-0ed8-41a1-a068-a02037ab6067 None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:12.387351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693396-0ed8-41a1-a068-a02037ab6067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:12.388759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693396-0ed8-41a1-a068-a02037ab6067 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:12.394071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 535/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:12.401802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:12.501314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79232665-8ef9-40c4-b447-31f735c38469 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:12.536189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea52a4-4776-40d6-a172-c5f51c03df45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:12.536791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea52a4-4776-40d6-a172-c5f51c03df45 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:12.537068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea52a4-4776-40d6-a172-c5f51c03df45 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:12.537337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea52a4-4776-40d6-a172-c5f51c03df45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:12.537984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea52a4-4776-40d6-a172-c5f51c03df45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:12.539342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea52a4-4776-40d6-a172-c5f51c03df45 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:12.563796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 524/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:00:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:13.190510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c8575-147c-4f05-aa12-98377d8d2780 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:13.191028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c8575-147c-4f05-aa12-98377d8d2780 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:13.191028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c8575-147c-4f05-aa12-98377d8d2780 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:13.192445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 536/1060] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:13.197419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f41c1-f2cb-4fa2-8ace-d3fa25c8be3c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:13.197799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f41c1-f2cb-4fa2-8ace-d3fa25c8be3c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:13.197962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f41c1-f2cb-4fa2-8ace-d3fa25c8be3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:13.216392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1d4f41c1-f2cb-4fa2-8ace-d3fa25c8be3c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:13.217591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1d4f41c1-f2cb-4fa2-8ace-d3fa25c8be3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:13.258910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:13.293609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 525/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:13.380095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2b193-f59a-4fed-a39f-8f2e6a6f6e6e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:13.380416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2b193-f59a-4fed-a39f-8f2e6a6f6e6e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:13.380657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2b193-f59a-4fed-a39f-8f2e6a6f6e6e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:13.382105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 537/1062] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:13.386936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae247c-c158-460d-87ef-c03f2cf1721a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:13.387229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae247c-c158-460d-87ef-c03f2cf1721a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:13.387503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae247c-c158-460d-87ef-c03f2cf1721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:13.406572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-efae247c-c158-460d-87ef-c03f2cf1721a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:13.407781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-efae247c-c158-460d-87ef-c03f2cf1721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:13.478281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 526/1063] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:13.556818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2b193-f59a-4fed-a39f-8f2e6a6f6e6e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:13.631148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:13.664140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2b193-f59a-4fed-a39f-8f2e6a6f6e6e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:13.698875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:13.699493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:13.699785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:13.700055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:13.710324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:13.715088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:13.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:13.720754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 538/1064] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:13.722588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae247c-c158-460d-87ef-c03f2cf1721a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:13.742441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:13.823343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:13.826375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae247c-c158-460d-87ef-c03f2cf1721a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:13.854219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd7192-5e05-45eb-ae7d-ae62691b8029 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:13.861595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:13.862197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:13.862478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:13.862745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:13.863401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:13.863697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:13.865083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:13.873519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 527/1065] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:00:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:13.879172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:13.888800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:13.895457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:13.896127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:13.896409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:13.896684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:13.907723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:13.912500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:13.913463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:13.914337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 539/1066] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 6 01:00:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:13.920357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:13.992073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543afe1-b1e1-4dea-a38d-095395a0b615 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.028152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d824edd-3380-42ca-8481-71e66cd80e53 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.028675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.029807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.030084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.030382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] PATH_INFO: `/v3/users/2fbc15c528c84b789758516262935bbd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.031736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] RBAC: Authorizing `identity:get_user(user_id=2fbc15c528c84b789758516262935bbd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.032914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.034690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 528/1067] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:13 2022] GET /identity/v3/users/2fbc15c528c84b789758516262935bbd => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:14.040012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.065993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.066607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.066891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.067169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.067863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:14.068174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.069531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.073035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:14.084180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 540/1068] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 6 01:00:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:14.084973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.143570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:14.154744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41930ee0-8561-456f-af4e-36ccc308e450 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.191694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.192599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.193029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.193446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.194298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e590a41-1016-46e8-9325-c05c2861c8d7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.195289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] RBAC: Authorizing `identity:get_project(project_id=7868e14f5def455999e86ee4ba979939)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.197141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.199224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 529/1069] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:14 2022] GET /identity/v3/projects/7868e14f5def455999e86ee4ba979939 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:14.204646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.234117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.235551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.236055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.236626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.248983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.250470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:14.266707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:14.267795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:14.268723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:14.268723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:14.268723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:14.269541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 541/1070] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:14.276133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.314307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:14.319033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a68f2-6fac-4e79-8798-282b48aef820 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.360092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ce424-69c2-448c-b756-20f300df766c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.360756 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ce424-69c2-448c-b756-20f300df766c None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.361037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ce424-69c2-448c-b756-20f300df766c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.361311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ce424-69c2-448c-b756-20f300df766c None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939/users/2fbc15c528c84b789758516262935bbd/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.364039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ce424-69c2-448c-b756-20f300df766c None admin] RBAC: Authorizing `identity:create_grant(project_id=7868e14f5def455999e86ee4ba979939, user_id=2fbc15c528c84b789758516262935bbd, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.365475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ce424-69c2-448c-b756-20f300df766c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.379623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 530/1071] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 01:00:14 2022] PUT /identity/v3/projects/7868e14f5def455999e86ee4ba979939/users/2fbc15c528c84b789758516262935bbd/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:00:14.386311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f761f-7fe9-4efa-b0f7-53f85a8747bd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.396793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:14.429795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfa00f-d497-4c70-b9ec-aaabc5737174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.430728 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfa00f-d497-4c70-b9ec-aaabc5737174 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.431225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfa00f-d497-4c70-b9ec-aaabc5737174 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.431664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfa00f-d497-4c70-b9ec-aaabc5737174 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.432516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfa00f-d497-4c70-b9ec-aaabc5737174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:14.432997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfa00f-d497-4c70-b9ec-aaabc5737174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.434647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfa00f-d497-4c70-b9ec-aaabc5737174 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.448312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 542/1072] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:14.454312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ce424-69c2-448c-b756-20f300df766c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.473573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018fa62-6221-49a0-8337-e1819b18e0fa None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.474146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018fa62-6221-49a0-8337-e1819b18e0fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.474630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018fa62-6221-49a0-8337-e1819b18e0fa None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.476437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 543/1073] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:14.484078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b40476-fc4a-4ef8-8ea0-5ebfeafbc238 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.484703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b40476-fc4a-4ef8-8ea0-5ebfeafbc238 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.485042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b40476-fc4a-4ef8-8ea0-5ebfeafbc238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.504761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-28b40476-fc4a-4ef8-8ea0-5ebfeafbc238 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:14.506087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28b40476-fc4a-4ef8-8ea0-5ebfeafbc238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:14.580524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ce424-69c2-448c-b756-20f300df766c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.589596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 544/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:14.617004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.617634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.617914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.618186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.625809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.627241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:00:14.638831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:14.639742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:14.640755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 531/1075] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 01:00:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:14.647472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b40476-fc4a-4ef8-8ea0-5ebfeafbc238 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.741916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:14.749912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b40476-fc4a-4ef8-8ea0-5ebfeafbc238 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.785582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.786193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.786469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.786736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.787450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:14.787787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.789185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.806612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 545/1076] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 01:00:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:14.812504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.837011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:14.862674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:14.925740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c798e1-6c81-4ae8-a187-6d58ff87ad89 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.962509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:14.962907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:14.963186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:14.963459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:14.969288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afebd1-f93f-4ca5-8e29-434c171cfbcd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:14.976325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:14.977767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:14.994219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:14.995495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:14.996964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:14.996964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:14.996964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:14.999911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 532/1077] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:15.001985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:15.008503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:15.009182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:15.009462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:15.009730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:15.036106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:15.075555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:15.084115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:15.086235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:15.107733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3c61f-d27c-49b0-9f42-76a7af0d13b3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:15.142161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:15.142983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:15.143433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:15.143708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:15.144387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:15.144692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:15.146060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:15.158940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 533/1078] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:15.164423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:15.193872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 546/1079] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 01:00:15.199663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:15.263003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:15.268630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa6f7f-a20b-438c-96a1-8bd8724690b7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:15.299663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:15.306786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:15.307439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:15.307695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:15.307998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:15.314360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab96996-b145-4e05-826b-9a425703344f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:15.314795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:15.316269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:00:15.329679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:15.330606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:15.331498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 534/1080] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:15.336518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:15.352696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:15.353324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:15.353600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:15.353873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:15.364090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:15.365454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:15.380999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:15.381994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:15.382905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:15.382905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:15.382905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:15.382905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 547/1081] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:15.387943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:15.443683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68b2d3-4301-4c3b-8f32-a9a8a5ccffd7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:15.484251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:15.484861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:15.485280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:15.485560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:15.486558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:15.487130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:15.489123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:15.491546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c65a68-2843-4aae-8c7b-cd61692bd8c3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:15.499155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:15.504474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 535/1082] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:15.510402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:15.537789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e26fb-996c-4948-9a8e-39a9350ffbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:15.538398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e26fb-996c-4948-9a8e-39a9350ffbec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:15.538767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e26fb-996c-4948-9a8e-39a9350ffbec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:15.538945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e26fb-996c-4948-9a8e-39a9350ffbec None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:15.539591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e26fb-996c-4948-9a8e-39a9350ffbec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:15.539928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e26fb-996c-4948-9a8e-39a9350ffbec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:15.541554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e26fb-996c-4948-9a8e-39a9350ffbec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:15.552581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 548/1083] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:15.580152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:15.639139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232005b-93fa-46c2-9f61-4d3ff9c04fd9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:15.674779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815cc2b-56a5-4ac8-ab1b-7e3bb46c596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:15.675460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815cc2b-56a5-4ac8-ab1b-7e3bb46c596d None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:15.675739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815cc2b-56a5-4ac8-ab1b-7e3bb46c596d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:15.676039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815cc2b-56a5-4ac8-ab1b-7e3bb46c596d None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/6b6caf1ad3784ddabda79ca0b1fc468d/roles/e412e312d7a0477abd23db933823bb46` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:15.678217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815cc2b-56a5-4ac8-ab1b-7e3bb46c596d None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=6b6caf1ad3784ddabda79ca0b1fc468d, role_id=e412e312d7a0477abd23db933823bb46)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:15.679685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815cc2b-56a5-4ac8-ab1b-7e3bb46c596d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:15.697728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 536/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 01:00:15 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/6b6caf1ad3784ddabda79ca0b1fc468d/roles/e412e312d7a0477abd23db933823bb46 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:00:16.302694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db785d7-a7f6-4227-8cad-88e5004d3900 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:16.303219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db785d7-a7f6-4227-8cad-88e5004d3900 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:16.303219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db785d7-a7f6-4227-8cad-88e5004d3900 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:16.304715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 549/1085] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:16.310055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66816934-daba-4be7-a677-86843f7aa77f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:16.310494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66816934-daba-4be7-a677-86843f7aa77f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:16.310727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66816934-daba-4be7-a677-86843f7aa77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:16.328504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-66816934-daba-4be7-a677-86843f7aa77f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:16.329677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-66816934-daba-4be7-a677-86843f7aa77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:16.393622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 537/1086] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:16.665409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db785d7-a7f6-4227-8cad-88e5004d3900 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:16.765234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db785d7-a7f6-4227-8cad-88e5004d3900 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:16.799859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:16.800488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:16.800770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:16.801035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:16.810762 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:16.815471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:16.816643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:16.817655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 550/1087] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:00:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:16.822550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66816934-daba-4be7-a677-86843f7aa77f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:16.862983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:16.919168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66816934-daba-4be7-a677-86843f7aa77f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:16.952716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:16.953318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:16.953600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:16.953873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:16.954527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:16.954829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:16.956242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:16.964065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 538/1088] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:16.969797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:16.999133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:17.069509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eef1c2-9d39-4620-a678-1fdc02363b62 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:17.102701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:17.103325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:17.103606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:17.103922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] PATH_INFO: `/v3/users/2fbc15c528c84b789758516262935bbd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:17.105053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] RBAC: Authorizing `identity:get_user(user_id=2fbc15c528c84b789758516262935bbd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:17.106466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:17.108087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 551/1089] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:16 2022] GET /identity/v3/users/2fbc15c528c84b789758516262935bbd => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:17.114241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:17.211365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c62db-8f79-4a95-80f5-7ddae994b140 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:17.212893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:17.244442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:17.245039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:17.245313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:17.245580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:17.317762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:17.319192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:17.411480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 539/1090] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:00:17.417462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:17.507008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:17.514311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c59c6-93a9-4d55-8a26-874bfb17811f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:17.552427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6552f-a0ee-4e69-b3f5-72824f7efc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:17.553515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6552f-a0ee-4e69-b3f5-72824f7efc5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:17.554056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6552f-a0ee-4e69-b3f5-72824f7efc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:17.554553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6552f-a0ee-4e69-b3f5-72824f7efc5d None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:17.556105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6552f-a0ee-4e69-b3f5-72824f7efc5d None admin] RBAC: Authorizing `identity:get_project(project_id=7868e14f5def455999e86ee4ba979939)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:17.557632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6552f-a0ee-4e69-b3f5-72824f7efc5d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:17.565880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 552/1091] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:17 2022] GET /identity/v3/projects/7868e14f5def455999e86ee4ba979939 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:17.567024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:17.569933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:17.590225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f032-6606-4495-a097-3be5b152a1c8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:17.590886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f032-6606-4495-a097-3be5b152a1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:17.590886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f032-6606-4495-a097-3be5b152a1c8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:17.592178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 553/1092] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 01:00:17.597056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea993ae-d2fa-479a-a140-ac941745c5a4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:17.597419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea993ae-d2fa-479a-a140-ac941745c5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:17.597700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea993ae-d2fa-479a-a140-ac941745c5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:17.616425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eea993ae-d2fa-479a-a140-ac941745c5a4 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:17.617622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eea993ae-d2fa-479a-a140-ac941745c5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:17.668094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbbfa9-2b66-441f-9ed4-1878f78f9bc1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:17.682761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 554/1093] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:17.700658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:17.701312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:17.701588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:17.701856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:17.702561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=7868e14f5def455999e86ee4ba979939, user.id=2fbc15c528c84b789758516262935bbd) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:17.702863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:17.704313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:17.709792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 540/1094] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:17 2022] GET /identity/v3/role_assignments?user.id=2fbc15c528c84b789758516262935bbd&scope.project.id=7868e14f5def455999e86ee4ba979939&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:17.953273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea993ae-d2fa-479a-a140-ac941745c5a4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:18.043256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:18.052880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea993ae-d2fa-479a-a140-ac941745c5a4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:18.086090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:18.086715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:18.086995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:18.087267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:18.097809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:18.102590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:18.103674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:18.104828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 555/1095] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 6 01:00:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:18.109662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:18.174123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:18.208575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c1f45-fabc-4a21-9607-60767bfafbbb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:18.240548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:18.241143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:18.241424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:18.241697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:18.242347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:18.242649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:18.244046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:18.251634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 541/1096] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 6 01:00:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:18.257339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:18.354261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed97a4-18d0-475c-a592-523f9a155104 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:18.386441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:18.387058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:18.387338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:18.387610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:18.397692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:18.399118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:18.414184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:18.420198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:18.421749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:18.421749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:18.421749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:18.421749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 556/1097] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:18.423340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:18.519641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae15d4e-2e01-4388-a212-d6f5261852b7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:18.544285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:18.552494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:18.553099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:18.553377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:18.553645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:18.554278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:18.554577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:18.555975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:18.567907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 542/1098] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:18.573559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:18.577957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:18.671344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f50f5-f698-4676-868e-4f54f1a7686c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:18.704917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:18.705549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:18.705830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:18.706107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:18.713427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:18.714868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:00:18.725686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:18.726706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:18.727645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 557/1099] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 6 01:00:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:18.732574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:18.833143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080f08a-4ab5-40d2-92a0-ae3ec9c16e73 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:18.866130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:18.866729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:18.867009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:18.867278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:18.868033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:18.868337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:18.869762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:18.886634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 543/1100] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 6 01:00:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:18.892618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:18.953037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:18.990113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b273a2f-5a00-4e9e-ae59-4746ec251c94 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:19.024447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:19.025051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.025330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.025600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.090514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:19.091954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:19.149520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:19.187913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 558/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 01:00:19.193665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:19.238860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:19.290211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e4a80-0546-4629-ab0b-dbc7f6ff19ce None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:19.323255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:19.323911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.324192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.324461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.334077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:19.335503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:19.350034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:19.350970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:00:19.351905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:19.351905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:00:19.351905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:19.351905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 544/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:00:19.356657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:19.451858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f67fd-676c-46dc-81e6-20ffa1cd64ce None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:19.487616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2c50c-df7f-4963-94d5-698f38d32b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:19.488295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c50c-df7f-4963-94d5-698f38d32b7b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.488576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c50c-df7f-4963-94d5-698f38d32b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.488848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c50c-df7f-4963-94d5-698f38d32b7b None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.489485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2c50c-df7f-4963-94d5-698f38d32b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:19.489785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2c50c-df7f-4963-94d5-698f38d32b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:19.491179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2c50c-df7f-4963-94d5-698f38d32b7b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:19.502125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 559/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:19.507872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:19.602925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:19.606737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d43f4-8dae-4b2e-9867-1b676d1ab044 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:19.610703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6956-95f6-483b-a183-462a2a81fac0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.610990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6956-95f6-483b-a183-462a2a81fac0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.611263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6956-95f6-483b-a183-462a2a81fac0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.612653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 560/1104] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 01:00:19.617592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071545cc-e31c-4bca-9312-8c4eeb8cf724 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.617876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071545cc-e31c-4bca-9312-8c4eeb8cf724 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.618155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071545cc-e31c-4bca-9312-8c4eeb8cf724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.635965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-071545cc-e31c-4bca-9312-8c4eeb8cf724 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:19.637149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-071545cc-e31c-4bca-9312-8c4eeb8cf724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:19.640961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:19.641592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.641877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.642153 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.648925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:19.650361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:19.654953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:00:19.662377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:19.663426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:19.664465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 545/1105] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:19.669451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:19.710018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 561/1106] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:19.766729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6b85a-26b4-4d40-a39f-e4cf705738ef None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:19.798947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:19.799540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.799858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.800131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.800841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:19.801142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:19.802618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:19.814083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 546/1107] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:19.820254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071545cc-e31c-4bca-9312-8c4eeb8cf724 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:19.916309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071545cc-e31c-4bca-9312-8c4eeb8cf724 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:19.952043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:19.952635 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:19.952915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:19.953186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:19.953897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] RBAC: Adding query filter params (role.id=e412e312d7a0477abd23db933823bb46, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=6b6caf1ad3784ddabda79ca0b1fc468d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:19.954204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:19.955614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:19.961272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 562/1108] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:19 2022] GET /identity/v3/role_assignments?user.id=6b6caf1ad3784ddabda79ca0b1fc468d&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=e412e312d7a0477abd23db933823bb46 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:20.008183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:20.028380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:20.113298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37b85e-08da-4f09-ba30-81f1df6c3ec4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:20.148414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebe340-40b8-48fe-8636-feffac09c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:20.149061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebe340-40b8-48fe-8636-feffac09c381 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:20.149363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebe340-40b8-48fe-8636-feffac09c381 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:20.149636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebe340-40b8-48fe-8636-feffac09c381 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:20.150730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebe340-40b8-48fe-8636-feffac09c381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:20.152252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebe340-40b8-48fe-8636-feffac09c381 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:20.153839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 547/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:20.159722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:20.180430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:20.273259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d65aa7-e8ae-49d4-bae3-c7bcde9f2aa5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:20.320551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c6a92-c109-4dd3-8a71-9e3b6206c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:20.321438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c6a92-c109-4dd3-8a71-9e3b6206c36f None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:20.321854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c6a92-c109-4dd3-8a71-9e3b6206c36f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:20.322272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c6a92-c109-4dd3-8a71-9e3b6206c36f None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:20.323262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c6a92-c109-4dd3-8a71-9e3b6206c36f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:20.325534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c6a92-c109-4dd3-8a71-9e3b6206c36f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:20.352842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 563/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 01:00:20 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:22.287607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e61858-0a50-48ca-9bd0-b37f0e026476 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:22.288163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e61858-0a50-48ca-9bd0-b37f0e026476 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:22.288285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e61858-0a50-48ca-9bd0-b37f0e026476 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:22.289515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 548/1111] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:22.295380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ffda-22ad-4112-b5d2-0c6722592681 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:22.295739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ffda-22ad-4112-b5d2-0c6722592681 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:22.295962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837ffda-22ad-4112-b5d2-0c6722592681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:22.313671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b837ffda-22ad-4112-b5d2-0c6722592681 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:22.314863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b837ffda-22ad-4112-b5d2-0c6722592681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:22.384900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 564/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:22.649079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e61858-0a50-48ca-9bd0-b37f0e026476 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:22.732514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:22.748424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e61858-0a50-48ca-9bd0-b37f0e026476 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:22.781625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:22.782243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:22.782524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:22.782794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] PATH_INFO: `/v3/roles/66704401130047e78e714312f45defd9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:22.784203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] RBAC: Authorizing `identity:get_role(role_id=66704401130047e78e714312f45defd9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:22.785571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:22.786865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 549/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:22 2022] GET /identity/v3/roles/66704401130047e78e714312f45defd9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:22.792715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ffda-22ad-4112-b5d2-0c6722592681 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:22.795785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:22.886983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837ffda-22ad-4112-b5d2-0c6722592681 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:22.920756 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:22.921373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:22.921653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:22.921922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] PATH_INFO: `/v3/users/bed13bbace7c4eb2bb917fa6e9b2c507` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:22.934384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] RBAC: Authorizing `identity:get_user(user_id=bed13bbace7c4eb2bb917fa6e9b2c507)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:22.936361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:22.937923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 565/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:22 2022] GET /identity/v3/users/bed13bbace7c4eb2bb917fa6e9b2c507 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:22.943613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:23.038002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f63c0-dcdf-4e2c-83be-ff7c1871441b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:23.071091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:23.071679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:23.071981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:23.072247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:23.094788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:23.135905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:23.137379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:23.229777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 550/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:00:23.235669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:23.295439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:23.332610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb620e-ef15-4fe4-b6ca-9ab2fc5d1939 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:23.364683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4ab95-7c64-4440-807a-dabce2d0831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:23.365289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4ab95-7c64-4440-807a-dabce2d0831d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:23.365567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4ab95-7c64-4440-807a-dabce2d0831d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:23.365837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4ab95-7c64-4440-807a-dabce2d0831d None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:23.366918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4ab95-7c64-4440-807a-dabce2d0831d None admin] RBAC: Authorizing `identity:get_project(project_id=7868e14f5def455999e86ee4ba979939)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:23.368357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4ab95-7c64-4440-807a-dabce2d0831d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:23.369931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 566/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:23 2022] GET /identity/v3/projects/7868e14f5def455999e86ee4ba979939 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:23.376549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:23.429037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:23.471022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad783ad-aa2c-47fc-81f2-e443b3821afb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:23.503676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aced4-cb3f-4928-b107-e4892421f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:23.504414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aced4-cb3f-4928-b107-e4892421f9aa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:23.504700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aced4-cb3f-4928-b107-e4892421f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:23.504975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aced4-cb3f-4928-b107-e4892421f9aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:23.505686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aced4-cb3f-4928-b107-e4892421f9aa None admin] RBAC: Adding query filter params (role.id=66704401130047e78e714312f45defd9, scope.project.id=7868e14f5def455999e86ee4ba979939, user.id=bed13bbace7c4eb2bb917fa6e9b2c507) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:23.505994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aced4-cb3f-4928-b107-e4892421f9aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:23.507410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961aced4-cb3f-4928-b107-e4892421f9aa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:23.512583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 551/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:23 2022] GET /identity/v3/role_assignments?user.id=bed13bbace7c4eb2bb917fa6e9b2c507&scope.project.id=7868e14f5def455999e86ee4ba979939&role.id=66704401130047e78e714312f45defd9 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:25.375586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423baf3-27d9-4892-af9c-7a2cebb5f34d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:25.376262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423baf3-27d9-4892-af9c-7a2cebb5f34d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:25.376262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423baf3-27d9-4892-af9c-7a2cebb5f34d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:25.377478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 567/1118] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:25.382467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5f735-3535-42dd-9c7d-6e79e98c3b25 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:25.382849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5f735-3535-42dd-9c7d-6e79e98c3b25 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:25.383031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5f735-3535-42dd-9c7d-6e79e98c3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:25.400739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fcc5f735-3535-42dd-9c7d-6e79e98c3b25 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:25.401912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fcc5f735-3535-42dd-9c7d-6e79e98c3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:25.466648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 552/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:25.741727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423baf3-27d9-4892-af9c-7a2cebb5f34d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:25.839356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423baf3-27d9-4892-af9c-7a2cebb5f34d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:25.866416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:25.872176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:25.872784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:25.873061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:25.873329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] PATH_INFO: `/v3/roles/66704401130047e78e714312f45defd9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:25.874685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] RBAC: Authorizing `identity:get_role(role_id=66704401130047e78e714312f45defd9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:25.876051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:25.877337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 568/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:25 2022] GET /identity/v3/roles/66704401130047e78e714312f45defd9 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:25.883054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5f735-3535-42dd-9c7d-6e79e98c3b25 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:25.930125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:25.984344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5f735-3535-42dd-9c7d-6e79e98c3b25 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:26.022445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:26.022994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:26.023273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:26.023541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] PATH_INFO: `/v3/users/bed13bbace7c4eb2bb917fa6e9b2c507` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:26.024674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] RBAC: Authorizing `identity:get_user(user_id=bed13bbace7c4eb2bb917fa6e9b2c507)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:26.026107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:26.027669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 553/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:25 2022] GET /identity/v3/users/bed13bbace7c4eb2bb917fa6e9b2c507 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:26.033490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:26.130151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea90d04-9958-4c61-9445-98f675ed7199 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:26.162564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b48a3-3f43-4956-b709-de5e1d30765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:26.163173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b48a3-3f43-4956-b709-de5e1d30765e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:26.163456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b48a3-3f43-4956-b709-de5e1d30765e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:26.163733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b48a3-3f43-4956-b709-de5e1d30765e None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:26.164864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b48a3-3f43-4956-b709-de5e1d30765e None admin] RBAC: Authorizing `identity:get_project(project_id=7868e14f5def455999e86ee4ba979939)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:26.166552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b48a3-3f43-4956-b709-de5e1d30765e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:26.168179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 569/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:26 2022] GET /identity/v3/projects/7868e14f5def455999e86ee4ba979939 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:26.173878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:26.268730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0b984-fb63-4091-ae40-ffaa0dc0b942 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:26.285006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:26.302748 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cfd14-5501-45c0-91b5-847ae17e0e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:26.303388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cfd14-5501-45c0-91b5-847ae17e0e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:26.303663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cfd14-5501-45c0-91b5-847ae17e0e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:26.303964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cfd14-5501-45c0-91b5-847ae17e0e20 None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939/users/bed13bbace7c4eb2bb917fa6e9b2c507/roles/66704401130047e78e714312f45defd9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:26.306046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cfd14-5501-45c0-91b5-847ae17e0e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=7868e14f5def455999e86ee4ba979939, user_id=bed13bbace7c4eb2bb917fa6e9b2c507, role_id=66704401130047e78e714312f45defd9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:26.307470 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cfd14-5501-45c0-91b5-847ae17e0e20 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:26.320095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 554/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 01:00:26 2022] PUT /identity/v3/projects/7868e14f5def455999e86ee4ba979939/users/bed13bbace7c4eb2bb917fa6e9b2c507/roles/66704401130047e78e714312f45defd9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:00:28.180042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d3cea-0852-40cf-883b-a1e4e9f3cb6e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:28.180604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d3cea-0852-40cf-883b-a1e4e9f3cb6e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:28.180604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d3cea-0852-40cf-883b-a1e4e9f3cb6e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:28.181922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 570/1124] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:28.187050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673efdd8-d352-43e3-b73f-7d77ad0a09ee None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:28.187442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673efdd8-d352-43e3-b73f-7d77ad0a09ee None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:28.187596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673efdd8-d352-43e3-b73f-7d77ad0a09ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:28.205177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-673efdd8-d352-43e3-b73f-7d77ad0a09ee None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:28.206342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-673efdd8-d352-43e3-b73f-7d77ad0a09ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:28.269763 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 555/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:28.537680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d3cea-0852-40cf-883b-a1e4e9f3cb6e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:28.602093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:28.636497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d3cea-0852-40cf-883b-a1e4e9f3cb6e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:28.670877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:28.671498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:28.671805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:28.672077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] PATH_INFO: `/v3/roles/66704401130047e78e714312f45defd9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:28.673448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] RBAC: Authorizing `identity:get_role(role_id=66704401130047e78e714312f45defd9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:28.674803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:28.676143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 571/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:28 2022] GET /identity/v3/roles/66704401130047e78e714312f45defd9 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:28.681954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673efdd8-d352-43e3-b73f-7d77ad0a09ee None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:28.775942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673efdd8-d352-43e3-b73f-7d77ad0a09ee None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:28.783452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:28.809707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:28.810300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:28.810574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:28.810843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] PATH_INFO: `/v3/users/bed13bbace7c4eb2bb917fa6e9b2c507` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:28.811958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] RBAC: Authorizing `identity:get_user(user_id=bed13bbace7c4eb2bb917fa6e9b2c507)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:28.813380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:28.814906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 556/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:28 2022] GET /identity/v3/users/bed13bbace7c4eb2bb917fa6e9b2c507 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:28.820548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:28.917068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d29654-30e5-478b-8ecc-66437b4d0722 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:28.948867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:28.949454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:28.949731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:28.950000 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:28.953088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:29.015132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:29.016592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:29.109855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 572/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:00:29.115958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:29.211448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5178c4d-7992-428e-961c-85cbd06578c0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:29.243253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e837ab5-5e14-4760-b02c-b585eae4992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:29.243892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837ab5-5e14-4760-b02c-b585eae4992f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:29.244173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837ab5-5e14-4760-b02c-b585eae4992f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:29.244444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837ab5-5e14-4760-b02c-b585eae4992f None admin] PATH_INFO: `/v3/projects/7868e14f5def455999e86ee4ba979939` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:29.245537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e837ab5-5e14-4760-b02c-b585eae4992f None admin] RBAC: Authorizing `identity:get_project(project_id=7868e14f5def455999e86ee4ba979939)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:29.246956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e837ab5-5e14-4760-b02c-b585eae4992f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:29.249054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 557/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:29 2022] GET /identity/v3/projects/7868e14f5def455999e86ee4ba979939 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:29.255066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:29.287271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:29.351971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391742f7-141b-471d-9c05-96c7d8cca658 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:29.384885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33c1d9-a51f-4d5f-8c20-f9db85236722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:29.385418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33c1d9-a51f-4d5f-8c20-f9db85236722 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:29.385693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33c1d9-a51f-4d5f-8c20-f9db85236722 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:29.385962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33c1d9-a51f-4d5f-8c20-f9db85236722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:29.386669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33c1d9-a51f-4d5f-8c20-f9db85236722 None admin] RBAC: Adding query filter params (role.id=66704401130047e78e714312f45defd9, scope.project.id=7868e14f5def455999e86ee4ba979939, user.id=bed13bbace7c4eb2bb917fa6e9b2c507) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:29.386970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33c1d9-a51f-4d5f-8c20-f9db85236722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:29.388401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33c1d9-a51f-4d5f-8c20-f9db85236722 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:29.393885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 573/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:29 2022] GET /identity/v3/role_assignments?user.id=bed13bbace7c4eb2bb917fa6e9b2c507&scope.project.id=7868e14f5def455999e86ee4ba979939&role.id=66704401130047e78e714312f45defd9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:31.290602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767a1e-b777-4291-8fc0-02d9d42dfdf6 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:31.291422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767a1e-b777-4291-8fc0-02d9d42dfdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:31.291422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767a1e-b777-4291-8fc0-02d9d42dfdf6 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:31.292790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 558/1131] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:31.298653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dd776-6159-4e46-91f9-ea6d95cc6c29 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:31.299054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dd776-6159-4e46-91f9-ea6d95cc6c29 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:31.299211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dd776-6159-4e46-91f9-ea6d95cc6c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:31.316918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1c1dd776-6159-4e46-91f9-ea6d95cc6c29 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:31.318110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c1dd776-6159-4e46-91f9-ea6d95cc6c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:31.381600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 574/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:31.650288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767a1e-b777-4291-8fc0-02d9d42dfdf6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:31.693723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:31.748738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767a1e-b777-4291-8fc0-02d9d42dfdf6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:31.782401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd740cc-bc99-4d50-be7e-bfda977c7b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:31.783016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd740cc-bc99-4d50-be7e-bfda977c7b3f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:31.783296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd740cc-bc99-4d50-be7e-bfda977c7b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:31.783568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd740cc-bc99-4d50-be7e-bfda977c7b3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:31.784695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd740cc-bc99-4d50-be7e-bfda977c7b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:31.786118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd740cc-bc99-4d50-be7e-bfda977c7b3f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:31.787645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 559/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:31.793649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dd776-6159-4e46-91f9-ea6d95cc6c29 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:31.819791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:31.889050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dd776-6159-4e46-91f9-ea6d95cc6c29 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:31.921405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bace9b-499f-454c-8157-62a7c505a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:31.921988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bace9b-499f-454c-8157-62a7c505a3f7 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:31.922263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bace9b-499f-454c-8157-62a7c505a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:31.922532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bace9b-499f-454c-8157-62a7c505a3f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:31.923170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bace9b-499f-454c-8157-62a7c505a3f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:31.924615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bace9b-499f-454c-8157-62a7c505a3f7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:31.951021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 575/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 01:00:31 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:00:33.878353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4cae-7071-45a1-9e99-417e93898d23 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:33.878851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4cae-7071-45a1-9e99-417e93898d23 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:33.878851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4cae-7071-45a1-9e99-417e93898d23 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:33.880208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 560/1135] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:33.885238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906c6b0-6a1b-496e-bee2-03354c65f172 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:33.885615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906c6b0-6a1b-496e-bee2-03354c65f172 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:33.885799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906c6b0-6a1b-496e-bee2-03354c65f172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:33.904713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9906c6b0-6a1b-496e-bee2-03354c65f172 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:33.905889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9906c6b0-6a1b-496e-bee2-03354c65f172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:33.970002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 576/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:34.238090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4cae-7071-45a1-9e99-417e93898d23 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:34.339563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4cae-7071-45a1-9e99-417e93898d23 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:34.349680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:34.372892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ed44b-4b39-43b6-bcc3-7c4f6ddcfb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:34.373501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ed44b-4b39-43b6-bcc3-7c4f6ddcfb92 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:34.373775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ed44b-4b39-43b6-bcc3-7c4f6ddcfb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:34.374041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ed44b-4b39-43b6-bcc3-7c4f6ddcfb92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:34.375116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ed44b-4b39-43b6-bcc3-7c4f6ddcfb92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:34.376577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ed44b-4b39-43b6-bcc3-7c4f6ddcfb92 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:34.378116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 561/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:00:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:34.383887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906c6b0-6a1b-496e-bee2-03354c65f172 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:34.395532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:34.479076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906c6b0-6a1b-496e-bee2-03354c65f172 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:34.511558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14e33f-c378-42ce-8856-f04f09197ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:34.512121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e33f-c378-42ce-8856-f04f09197ff4 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:34.512401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e33f-c378-42ce-8856-f04f09197ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:34.512699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e33f-c378-42ce-8856-f04f09197ff4 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:34.513922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14e33f-c378-42ce-8856-f04f09197ff4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:34.515349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14e33f-c378-42ce-8856-f04f09197ff4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:34.535244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 577/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 01:00:34 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:36.481794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938fcf-cfda-498f-b23e-10d1f5f2301a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:36.482346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938fcf-cfda-498f-b23e-10d1f5f2301a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:36.482346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938fcf-cfda-498f-b23e-10d1f5f2301a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:36.483648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 562/1139] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:36.488769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae2333-4e5b-40f1-9ee4-d848fc2c44be None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:36.489181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae2333-4e5b-40f1-9ee4-d848fc2c44be None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:36.489301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae2333-4e5b-40f1-9ee4-d848fc2c44be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:36.507329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a5ae2333-4e5b-40f1-9ee4-d848fc2c44be None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:36.508543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a5ae2333-4e5b-40f1-9ee4-d848fc2c44be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:36.566282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:36.574398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 578/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:00:36.846457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938fcf-cfda-498f-b23e-10d1f5f2301a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:36.944649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938fcf-cfda-498f-b23e-10d1f5f2301a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:36.976516 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:36.977133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:36.977415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:36.977689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:36.987952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:36.993170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:36.994311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:36.995203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 563/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:00:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:37.000723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae2333-4e5b-40f1-9ee4-d848fc2c44be None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:37.098709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae2333-4e5b-40f1-9ee4-d848fc2c44be None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:37.133415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:37.134003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:37.134278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:37.134546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:37.135230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:37.135527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:37.136909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:37.144535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 579/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:00:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:37.151306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:37.190331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:37.247154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777cd05-fef1-4787-8214-97efee839770 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:37.279023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:37.279628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:37.279934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:37.280200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] PATH_INFO: `/v3/users/55473410145d4728b729030ccab55f3c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:37.292683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] RBAC: Authorizing `identity:get_user(user_id=55473410145d4728b729030ccab55f3c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:37.294705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:37.296273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 564/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:37 2022] GET /identity/v3/users/55473410145d4728b729030ccab55f3c => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:37.302169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:37.360128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:37.401139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436137aa-0d47-47b5-a8f5-9f51bf3c38ec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:37.435156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:37.435781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:37.436062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:37.436333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:37.499888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:37.501316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:37.555097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:37.596697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 580/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 01:00:37.602597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:37.692870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:37.699217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104f5c7-78d3-49a1-9356-661afc3868ac None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:37.732899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9bef9-2499-4ccf-8fc1-79c0e877b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:37.733522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9bef9-2499-4ccf-8fc1-79c0e877b7fb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:37.733800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9bef9-2499-4ccf-8fc1-79c0e877b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:37.734072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9bef9-2499-4ccf-8fc1-79c0e877b7fb None admin] PATH_INFO: `/v3/projects/800ddff102a3422aa8996c2e3062d2f0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:37.735167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9bef9-2499-4ccf-8fc1-79c0e877b7fb None admin] RBAC: Authorizing `identity:get_project(project_id=800ddff102a3422aa8996c2e3062d2f0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:37.736622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9bef9-2499-4ccf-8fc1-79c0e877b7fb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:37.738235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 565/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:37 2022] GET /identity/v3/projects/800ddff102a3422aa8996c2e3062d2f0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:37.744345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:37.839489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e4f2a-a124-46c9-8606-08789885cb5d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:37.872039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5264ea2-f5d4-4d0b-bcb6-f92adeabfd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:37.872639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264ea2-f5d4-4d0b-bcb6-f92adeabfd97 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:37.872916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264ea2-f5d4-4d0b-bcb6-f92adeabfd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:37.873186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264ea2-f5d4-4d0b-bcb6-f92adeabfd97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:37.873894 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5264ea2-f5d4-4d0b-bcb6-f92adeabfd97 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=800ddff102a3422aa8996c2e3062d2f0, user.id=55473410145d4728b729030ccab55f3c) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:37.874201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5264ea2-f5d4-4d0b-bcb6-f92adeabfd97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:37.875609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5264ea2-f5d4-4d0b-bcb6-f92adeabfd97 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:37.880768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 581/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:37 2022] GET /identity/v3/role_assignments?user.id=55473410145d4728b729030ccab55f3c&scope.project.id=800ddff102a3422aa8996c2e3062d2f0&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:39.740853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79625-3ec4-4b6b-9ebd-d343f0677f70 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:39.741373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79625-3ec4-4b6b-9ebd-d343f0677f70 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:39.741373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc79625-3ec4-4b6b-9ebd-d343f0677f70 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:39.742720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 566/1147] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:39.748266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c806-b2cc-48f5-bafa-48f529e99430 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:39.748632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c806-b2cc-48f5-bafa-48f529e99430 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:39.748819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c806-b2cc-48f5-bafa-48f529e99430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:39.766939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-61b4c806-b2cc-48f5-bafa-48f529e99430 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:39.768145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-61b4c806-b2cc-48f5-bafa-48f529e99430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:39.784122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:39.833622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 582/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:40.102857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc79625-3ec4-4b6b-9ebd-d343f0677f70 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:40.200516 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc79625-3ec4-4b6b-9ebd-d343f0677f70 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:40.234538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:40.235146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:40.235423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:40.235691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:40.245275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:40.249998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:40.251102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:40.252169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 567/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:40.256992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c806-b2cc-48f5-bafa-48f529e99430 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:40.352642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c806-b2cc-48f5-bafa-48f529e99430 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:40.385908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:40.386449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:40.386727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:40.386999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:40.387676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:40.388004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:40.389355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:40.397014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 583/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:00:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:40.402785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:40.428511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:40.499723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011938-e4bc-4081-9dfe-5afa5bff2369 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:40.532174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:40.532776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:40.533052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:40.533321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] PATH_INFO: `/v3/users/55473410145d4728b729030ccab55f3c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:40.534403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=55473410145d4728b729030ccab55f3c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:40.535844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:40.537362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 568/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:40 2022] GET /identity/v3/users/55473410145d4728b729030ccab55f3c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:40.543350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:40.559406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:40.640166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e6894-92ac-4f5d-a5f8-89aa7ac18eab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:40.672222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e377b55-443f-4219-be71-ea1bc55c8216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:40.672838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e377b55-443f-4219-be71-ea1bc55c8216 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:40.673116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e377b55-443f-4219-be71-ea1bc55c8216 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:40.673387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e377b55-443f-4219-be71-ea1bc55c8216 None admin] PATH_INFO: `/v3/projects/800ddff102a3422aa8996c2e3062d2f0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:40.674478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e377b55-443f-4219-be71-ea1bc55c8216 None admin] RBAC: Authorizing `identity:get_project(project_id=800ddff102a3422aa8996c2e3062d2f0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:40.675921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e377b55-443f-4219-be71-ea1bc55c8216 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:40.677523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 584/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:40 2022] GET /identity/v3/projects/800ddff102a3422aa8996c2e3062d2f0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:40.683416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:40.768925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:40.777818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6ead9-26db-429b-8045-008a1ea1f2f4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:40.810946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373d60e-beca-433e-af0d-113107fe9c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:40.811587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373d60e-beca-433e-af0d-113107fe9c7a None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:40.811899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373d60e-beca-433e-af0d-113107fe9c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:40.812199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373d60e-beca-433e-af0d-113107fe9c7a None admin] PATH_INFO: `/v3/projects/800ddff102a3422aa8996c2e3062d2f0/users/55473410145d4728b729030ccab55f3c/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:40.814050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373d60e-beca-433e-af0d-113107fe9c7a None admin] RBAC: Authorizing `identity:create_grant(project_id=800ddff102a3422aa8996c2e3062d2f0, user_id=55473410145d4728b729030ccab55f3c, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:40.815468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373d60e-beca-433e-af0d-113107fe9c7a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:40.828127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 569/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 01:00:40 2022] PUT /identity/v3/projects/800ddff102a3422aa8996c2e3062d2f0/users/55473410145d4728b729030ccab55f3c/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:00:42.694424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb61216-083e-4e91-b8b7-e13f28d89fa4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:42.694921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb61216-083e-4e91-b8b7-e13f28d89fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:42.695038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb61216-083e-4e91-b8b7-e13f28d89fa4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:42.696320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 585/1154] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:42.701638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2097a7-0153-46aa-85d5-5de39ff31a02 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:42.702050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2097a7-0153-46aa-85d5-5de39ff31a02 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:42.702191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2097a7-0153-46aa-85d5-5de39ff31a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:42.720277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7c2097a7-0153-46aa-85d5-5de39ff31a02 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:42.721451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7c2097a7-0153-46aa-85d5-5de39ff31a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:42.785637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 570/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:43.056146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb61216-083e-4e91-b8b7-e13f28d89fa4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:43.135023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:43.154424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb61216-083e-4e91-b8b7-e13f28d89fa4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:43.186775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:43.187389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:43.187667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:43.187968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:43.198516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:43.203945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:43.205125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:43.206036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 586/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:00:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:43.211154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2097a7-0153-46aa-85d5-5de39ff31a02 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:43.288798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:43.307068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2097a7-0153-46aa-85d5-5de39ff31a02 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:43.339910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:43.340510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:43.340817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:43.341089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:43.341740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:43.342042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:43.343401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:43.351072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 571/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:00:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:43.356665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:43.452953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608baf5-1afe-4285-ae46-7f3f4c2a2b0c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:43.484835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:43.485443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:43.485724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:43.485998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] PATH_INFO: `/v3/users/55473410145d4728b729030ccab55f3c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:43.487094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] RBAC: Authorizing `identity:get_user(user_id=55473410145d4728b729030ccab55f3c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:43.488545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:43.490071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 587/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:43 2022] GET /identity/v3/users/55473410145d4728b729030ccab55f3c => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:43.495885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:43.592588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803ab35-d31c-45dd-8db9-e6a84897f1b9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:43.625476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:43.626070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:43.626348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:43.626619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:43.642315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:43.695951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:43.697396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:43.789395 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 572/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:00:43.796117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:43.808397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:43.892724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57c95f-8608-44c0-a945-ca61264e07c8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:43.925483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462315f7-332c-4e1c-a1f9-7cc3b6576bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:43.926093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462315f7-332c-4e1c-a1f9-7cc3b6576bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:43.926370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462315f7-332c-4e1c-a1f9-7cc3b6576bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:43.926638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462315f7-332c-4e1c-a1f9-7cc3b6576bb3 None admin] PATH_INFO: `/v3/projects/800ddff102a3422aa8996c2e3062d2f0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:43.927720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462315f7-332c-4e1c-a1f9-7cc3b6576bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=800ddff102a3422aa8996c2e3062d2f0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:43.929169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462315f7-332c-4e1c-a1f9-7cc3b6576bb3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:43.930740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 588/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:43 2022] GET /identity/v3/projects/800ddff102a3422aa8996c2e3062d2f0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:43.936837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:43.979387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:44.048569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391c125-f971-4fe0-92ff-f8ec239752ef None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:44.080173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ff353-1ffa-4709-81e6-a8870ab66458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:44.080762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ff353-1ffa-4709-81e6-a8870ab66458 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:44.081038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ff353-1ffa-4709-81e6-a8870ab66458 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:44.081306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ff353-1ffa-4709-81e6-a8870ab66458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:44.082009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ff353-1ffa-4709-81e6-a8870ab66458 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=800ddff102a3422aa8996c2e3062d2f0, user.id=55473410145d4728b729030ccab55f3c) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:44.082309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ff353-1ffa-4709-81e6-a8870ab66458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:44.083726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ff353-1ffa-4709-81e6-a8870ab66458 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:44.089203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 573/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:43 2022] GET /identity/v3/role_assignments?user.id=55473410145d4728b729030ccab55f3c&scope.project.id=800ddff102a3422aa8996c2e3062d2f0&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:45.985091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca50b6-9d49-4887-af5f-05356158189b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:45.985640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca50b6-9d49-4887-af5f-05356158189b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:45.985640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca50b6-9d49-4887-af5f-05356158189b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:45.986957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 589/1162] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:45.992010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c75ce3-a871-404a-b3aa-1eefa57f6c9b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:45.992432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c75ce3-a871-404a-b3aa-1eefa57f6c9b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:45.992555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c75ce3-a871-404a-b3aa-1eefa57f6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:46.010148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d3c75ce3-a871-404a-b3aa-1eefa57f6c9b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:46.011305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3c75ce3-a871-404a-b3aa-1eefa57f6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:46.077789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 574/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:46.346994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca50b6-9d49-4887-af5f-05356158189b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:46.423695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:46.446072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca50b6-9d49-4887-af5f-05356158189b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:46.478391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:46.478978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:46.479250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:46.479514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:46.542958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:46.544900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:46.623878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:46.645518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 590/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:00:46.651534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c75ce3-a871-404a-b3aa-1eefa57f6c9b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:46.690076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:46.746056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c75ce3-a871-404a-b3aa-1eefa57f6c9b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:46.778695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599b8a-8f2e-415b-97b5-78190528acb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:46.779315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599b8a-8f2e-415b-97b5-78190528acb1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:46.779591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599b8a-8f2e-415b-97b5-78190528acb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:46.779882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599b8a-8f2e-415b-97b5-78190528acb1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:46.786258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8599b8a-8f2e-415b-97b5-78190528acb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:46.787694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8599b8a-8f2e-415b-97b5-78190528acb1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d8599b8a-8f2e-415b-97b5-78190528acb1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:46.799456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:46.800653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:00:46.801513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 575/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 01:00:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:46.806608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:46.900964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93a27b-f6ad-4318-a964-d3149faa8c6f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:46.938769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8b0d7-78a3-4642-b7bb-1935080761c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:46.939613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b0d7-78a3-4642-b7bb-1935080761c2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:46.940122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b0d7-78a3-4642-b7bb-1935080761c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:46.940527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b0d7-78a3-4642-b7bb-1935080761c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:46.941465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8b0d7-78a3-4642-b7bb-1935080761c2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:46.941917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8b0d7-78a3-4642-b7bb-1935080761c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:46.944085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8b0d7-78a3-4642-b7bb-1935080761c2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:46.955103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 591/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 6 01:00:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:00:48.812829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcee0e1-b49b-4b2b-a5ee-23c4a0a887b0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:48.813353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcee0e1-b49b-4b2b-a5ee-23c4a0a887b0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:48.813353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcee0e1-b49b-4b2b-a5ee-23c4a0a887b0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:48.814704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 576/1167] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:48.819652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961864ce-c9b9-4caf-a4dc-d7047411f756 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:48.820035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961864ce-c9b9-4caf-a4dc-d7047411f756 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:48.820232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961864ce-c9b9-4caf-a4dc-d7047411f756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:48.837932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-961864ce-c9b9-4caf-a4dc-d7047411f756 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:48.839039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-961864ce-c9b9-4caf-a4dc-d7047411f756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:48.873974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:48.904163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 592/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:49.167336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcee0e1-b49b-4b2b-a5ee-23c4a0a887b0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:49.262291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:49.267327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcee0e1-b49b-4b2b-a5ee-23c4a0a887b0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:49.301717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512eac-3f73-4c70-943f-e7fd6649d9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:49.302308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512eac-3f73-4c70-943f-e7fd6649d9d1 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:49.302584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512eac-3f73-4c70-943f-e7fd6649d9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:49.302852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512eac-3f73-4c70-943f-e7fd6649d9d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:49.303514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512eac-3f73-4c70-943f-e7fd6649d9d1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:49.304997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512eac-3f73-4c70-943f-e7fd6649d9d1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:49.320008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 577/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Tue Sep 6 01:00:49 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:51.224868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c689a-260e-4621-9fa7-9197383f80de None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:51.225389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c689a-260e-4621-9fa7-9197383f80de None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:51.225574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c689a-260e-4621-9fa7-9197383f80de None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:51.226753 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 593/1170] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:51.231778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277d85f-8ae6-4445-a088-4281df9ba230 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:51.232189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277d85f-8ae6-4445-a088-4281df9ba230 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:51.232320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277d85f-8ae6-4445-a088-4281df9ba230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:51.249811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4277d85f-8ae6-4445-a088-4281df9ba230 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:51.250972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4277d85f-8ae6-4445-a088-4281df9ba230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:51.315180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 578/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:51.590589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c689a-260e-4621-9fa7-9197383f80de None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:51.687804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c689a-260e-4621-9fa7-9197383f80de None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:51.720217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41bf09-3fbd-4b2f-ae0c-32d0ad419bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:51.720825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41bf09-3fbd-4b2f-ae0c-32d0ad419bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:51.721105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41bf09-3fbd-4b2f-ae0c-32d0ad419bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:51.721379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41bf09-3fbd-4b2f-ae0c-32d0ad419bc4 None admin] PATH_INFO: `/v3/domains/7918fb34e8774d6eb782b9827208cc49` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:51.731276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41bf09-3fbd-4b2f-ae0c-32d0ad419bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7918fb34e8774d6eb782b9827208cc49)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:51.732733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41bf09-3fbd-4b2f-ae0c-32d0ad419bc4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:51.734268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 594/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Sep 6 01:00:51 2022] GET /identity/v3/domains/7918fb34e8774d6eb782b9827208cc49 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:51.740050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277d85f-8ae6-4445-a088-4281df9ba230 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:51.824795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:51.837372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277d85f-8ae6-4445-a088-4281df9ba230 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:51.869258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86cdc7-4ce4-4712-971b-1068ca6c9771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:51.869847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86cdc7-4ce4-4712-971b-1068ca6c9771 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:51.870122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86cdc7-4ce4-4712-971b-1068ca6c9771 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:51.870389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86cdc7-4ce4-4712-971b-1068ca6c9771 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:51.871028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86cdc7-4ce4-4712-971b-1068ca6c9771 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:51.873076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86cdc7-4ce4-4712-971b-1068ca6c9771 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:51.897529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 579/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 01:00:51 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:53.878749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1d94-16c1-40b4-8738-4b812a6de9b1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:53.879263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1d94-16c1-40b4-8738-4b812a6de9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:53.879263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1d94-16c1-40b4-8738-4b812a6de9b1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:53.880702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 595/1174] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:53.885791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04719f-eda2-44ce-8cc7-75823d01db14 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:53.886192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04719f-eda2-44ce-8cc7-75823d01db14 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:53.886369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04719f-eda2-44ce-8cc7-75823d01db14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:53.904520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0f04719f-eda2-44ce-8cc7-75823d01db14 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:53.905693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0f04719f-eda2-44ce-8cc7-75823d01db14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:53.970222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 580/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:54.251529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b1d94-16c1-40b4-8738-4b812a6de9b1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:54.283173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:54.352847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b1d94-16c1-40b4-8738-4b812a6de9b1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:54.385270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e23851-8cf8-4540-889d-ab4575a5e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:54.385887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23851-8cf8-4540-889d-ab4575a5e985 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:54.386167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23851-8cf8-4540-889d-ab4575a5e985 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:54.386438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23851-8cf8-4540-889d-ab4575a5e985 None admin] PATH_INFO: `/v3/domains/7918fb34e8774d6eb782b9827208cc49` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:54.388068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e23851-8cf8-4540-889d-ab4575a5e985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7918fb34e8774d6eb782b9827208cc49)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:54.389480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e23851-8cf8-4540-889d-ab4575a5e985 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:54.391018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 596/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Sep 6 01:00:54 2022] GET /identity/v3/domains/7918fb34e8774d6eb782b9827208cc49 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:54.396702 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04719f-eda2-44ce-8cc7-75823d01db14 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:54.463348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:54.494160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04719f-eda2-44ce-8cc7-75823d01db14 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:54.527553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeedde1-ff46-42fa-9642-d481c8567774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:54.528150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeedde1-ff46-42fa-9642-d481c8567774 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:54.528432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeedde1-ff46-42fa-9642-d481c8567774 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:54.528772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeedde1-ff46-42fa-9642-d481c8567774 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:54.529425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeedde1-ff46-42fa-9642-d481c8567774 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:54.530853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeedde1-ff46-42fa-9642-d481c8567774 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:54.551263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 581/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 01:00:54 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:00:56.471802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3c07-5342-4b8d-9602-16679df0a2b9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:56.472453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3c07-5342-4b8d-9602-16679df0a2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:56.472453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e3c07-5342-4b8d-9602-16679df0a2b9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:56.473682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 597/1178] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:56.478838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3599d-a6f7-45b8-86b6-75dcc67e7058 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:56.479279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3599d-a6f7-45b8-86b6-75dcc67e7058 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:56.479386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3599d-a6f7-45b8-86b6-75dcc67e7058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:56.498041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-54f3599d-a6f7-45b8-86b6-75dcc67e7058 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:56.499204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-54f3599d-a6f7-45b8-86b6-75dcc67e7058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:56.562508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 582/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:00:56.831224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e3c07-5342-4b8d-9602-16679df0a2b9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:56.925856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:56.929960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e3c07-5342-4b8d-9602-16679df0a2b9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:56.961758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:56.962362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:56.962638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:56.962904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:56.975472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:00:56.980338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:00:56.982097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:00:56.983025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 598/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:00:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:00:56.988017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3599d-a6f7-45b8-86b6-75dcc67e7058 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:57.041479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:57.083356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3599d-a6f7-45b8-86b6-75dcc67e7058 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:57.115900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:57.116486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:57.116760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:57.117028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:57.117673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:57.117969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:57.119326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:57.126908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 583/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:57.132685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:57.237666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b86383-44b0-46d6-806a-960243234bab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:57.270038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:57.270651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:57.270930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:57.271203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] PATH_INFO: `/v3/users/29ea11eb17ec43978396819fc420fcce` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:57.284234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] RBAC: Authorizing `identity:get_user(user_id=29ea11eb17ec43978396819fc420fcce)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:57.285664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:57.287205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 599/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:00:57 2022] GET /identity/v3/users/29ea11eb17ec43978396819fc420fcce => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:57.293246 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:57.388507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085167f5-022f-4fd3-b8b2-f683f57fb5a6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:57.404943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:57.421784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:57.422386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:57.422668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:57.422939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:57.486954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:57.488421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:57.585101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 584/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:00:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:00:57.590880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:57.609098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:57.687773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abe549-fb8e-49fb-a533-1f5479c333ff None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:57.720334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb998e5-f98e-48df-82cb-03cbb26e3502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:57.720946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb998e5-f98e-48df-82cb-03cbb26e3502 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:57.721228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb998e5-f98e-48df-82cb-03cbb26e3502 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:57.721504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb998e5-f98e-48df-82cb-03cbb26e3502 None admin] PATH_INFO: `/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:57.722600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb998e5-f98e-48df-82cb-03cbb26e3502 None admin] RBAC: Authorizing `identity:get_project(project_id=6ab2f376c7f043d5a8210e0e3fff03cb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:57.724045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb998e5-f98e-48df-82cb-03cbb26e3502 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:57.725639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 600/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:00:57 2022] GET /identity/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:57.731576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:00:57.749400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:00:57.826828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699fd24-ef2f-4b52-af2b-fce753be2edf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:00:57.858916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046a4d9-73a0-41df-ae74-bdd7d94fc328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:00:57.859494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a4d9-73a0-41df-ae74-bdd7d94fc328 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:57.859789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a4d9-73a0-41df-ae74-bdd7d94fc328 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:57.860062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046a4d9-73a0-41df-ae74-bdd7d94fc328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:57.860767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a4d9-73a0-41df-ae74-bdd7d94fc328 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=6ab2f376c7f043d5a8210e0e3fff03cb, user.id=29ea11eb17ec43978396819fc420fcce) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:00:57.861065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a4d9-73a0-41df-ae74-bdd7d94fc328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:00:57.862478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046a4d9-73a0-41df-ae74-bdd7d94fc328 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:00:57.867558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 585/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:00:57 2022] GET /identity/v3/role_assignments?user.id=29ea11eb17ec43978396819fc420fcce&scope.project.id=6ab2f376c7f043d5a8210e0e3fff03cb&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:00:59.775350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24e7e-c4d9-4b61-8236-0f18121c110f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:59.775908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24e7e-c4d9-4b61-8236-0f18121c110f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:59.775908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24e7e-c4d9-4b61-8236-0f18121c110f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:59.777232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 601/1186] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:00:59.782372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fde34a-4bbc-45fb-8c0f-71320872c4ac None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:00:59.782798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fde34a-4bbc-45fb-8c0f-71320872c4ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:00:59.782916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fde34a-4bbc-45fb-8c0f-71320872c4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:00:59.800697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-84fde34a-4bbc-45fb-8c0f-71320872c4ac None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:00:59.801853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-84fde34a-4bbc-45fb-8c0f-71320872c4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:00:59.864587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 586/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:00:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:01:00.135200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24e7e-c4d9-4b61-8236-0f18121c110f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:00.217535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:00.234424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24e7e-c4d9-4b61-8236-0f18121c110f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:00.267203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:00.267845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:00.268136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:00.268409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:00.278130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:01:00.282882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:01:00.283838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:01:00.284736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 602/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:01:00.289792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fde34a-4bbc-45fb-8c0f-71320872c4ac None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:00.304120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:00.385576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fde34a-4bbc-45fb-8c0f-71320872c4ac None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:00.417834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:00.419009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:00.419282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:00.419544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:00.420213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:01:00.420509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:00.421860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:00.429461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 587/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:01:00.435112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:00.532282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10ae32-cf53-4fb1-a80c-3bce1d2bde00 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:00.565283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:00.565877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:00.566159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:00.566436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] PATH_INFO: `/v3/users/29ea11eb17ec43978396819fc420fcce` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:00.567545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] RBAC: Authorizing `identity:get_user(user_id=29ea11eb17ec43978396819fc420fcce)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:00.569007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:00.570555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 603/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:01:00 2022] GET /identity/v3/users/29ea11eb17ec43978396819fc420fcce => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:01:00.576328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:00.648196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:00.672655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa92010-e7de-4e5c-86d2-c17fc6dceb83 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:00.703805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25ed0f-f876-40a4-88a0-734b8596668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:00.704426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25ed0f-f876-40a4-88a0-734b8596668d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:00.704702 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25ed0f-f876-40a4-88a0-734b8596668d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:00.704971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25ed0f-f876-40a4-88a0-734b8596668d None admin] PATH_INFO: `/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:00.706070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25ed0f-f876-40a4-88a0-734b8596668d None admin] RBAC: Authorizing `identity:get_project(project_id=6ab2f376c7f043d5a8210e0e3fff03cb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:00.707486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25ed0f-f876-40a4-88a0-734b8596668d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:00.709122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 588/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:01:00 2022] GET /identity/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:01:00.717472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:00.734230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:00.814306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b4173-95fc-4a04-b807-8e5a1c8adf1b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:00.846664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ecc61-3fdd-4cb4-bc74-1f02e39d4139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:00.847301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ecc61-3fdd-4cb4-bc74-1f02e39d4139 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:00.847580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ecc61-3fdd-4cb4-bc74-1f02e39d4139 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:00.847881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ecc61-3fdd-4cb4-bc74-1f02e39d4139 None admin] PATH_INFO: `/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb/users/29ea11eb17ec43978396819fc420fcce/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:00.849741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ecc61-3fdd-4cb4-bc74-1f02e39d4139 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ab2f376c7f043d5a8210e0e3fff03cb, user_id=29ea11eb17ec43978396819fc420fcce, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:00.851154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ecc61-3fdd-4cb4-bc74-1f02e39d4139 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:00.863862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 604/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 01:01:00 2022] PUT /identity/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb/users/29ea11eb17ec43978396819fc420fcce/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:01:02.727817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514b7f-1472-40c7-bfc2-62630b6f02f5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:02.728337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514b7f-1472-40c7-bfc2-62630b6f02f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:02.728443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514b7f-1472-40c7-bfc2-62630b6f02f5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:02.729687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 589/1193] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:01:02.734902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d2b63-e156-4f8a-9a21-7a7ed1273728 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:02.735272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d2b63-e156-4f8a-9a21-7a7ed1273728 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:02.735455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d2b63-e156-4f8a-9a21-7a7ed1273728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:02.753616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-394d2b63-e156-4f8a-9a21-7a7ed1273728 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:01:02.754793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-394d2b63-e156-4f8a-9a21-7a7ed1273728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:01:02.818467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 605/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:01:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:01:03.088285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb514b7f-1472-40c7-bfc2-62630b6f02f5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:03.186357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb514b7f-1472-40c7-bfc2-62630b6f02f5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:03.218271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:03.218879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:03.219157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:03.219427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:03.229121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:01:03.234401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:01:03.235496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:01:03.236378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 590/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:01:03.241255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d2b63-e156-4f8a-9a21-7a7ed1273728 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:03.244296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:03.336388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d2b63-e156-4f8a-9a21-7a7ed1273728 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:03.369638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:03.370185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:03.370465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:03.370738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:03.371996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:01:03.372301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:03.373648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:03.381297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 606/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:01:03.386926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:03.390292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:03.487521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5681-798f-4669-b9d3-b752f07864aa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:03.519644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:03.520399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:03.520711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:03.522092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] PATH_INFO: `/v3/users/29ea11eb17ec43978396819fc420fcce` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:03.522744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] RBAC: Authorizing `identity:get_user(user_id=29ea11eb17ec43978396819fc420fcce)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:03.523952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:03.531192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 591/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 6 01:01:03 2022] GET /identity/v3/users/29ea11eb17ec43978396819fc420fcce => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:01:03.532088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:03.595895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:03.630459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eee5c-eb10-413c-993d-0918424991f4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:03.662729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:03.663323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:03.663603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:03.663908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:03.733307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:03.734660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:03.795104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:03.829468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 607/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:01:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:01:03.835409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:03.901484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:03.931494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac42bdc-a640-4aa6-81a4-4ede9a5c2803 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:03.963477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67350b8e-e71e-4bf6-87af-62411352deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:03.964111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67350b8e-e71e-4bf6-87af-62411352deeb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:03.964386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67350b8e-e71e-4bf6-87af-62411352deeb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:03.964656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67350b8e-e71e-4bf6-87af-62411352deeb None admin] PATH_INFO: `/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:03.966306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67350b8e-e71e-4bf6-87af-62411352deeb None admin] RBAC: Authorizing `identity:get_project(project_id=6ab2f376c7f043d5a8210e0e3fff03cb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:03.967714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67350b8e-e71e-4bf6-87af-62411352deeb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:03.969319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 592/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 6 01:01:03 2022] GET /identity/v3/projects/6ab2f376c7f043d5a8210e0e3fff03cb => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:01:03.975411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:01:04.071223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9533c-3f59-4e00-a052-8cbbdbfcca88 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:01:04.104291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677738a-eb34-42b4-9ea1-1b472aa5c03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:01:04.104887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677738a-eb34-42b4-9ea1-1b472aa5c03b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:04.105168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677738a-eb34-42b4-9ea1-1b472aa5c03b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:04.105441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677738a-eb34-42b4-9ea1-1b472aa5c03b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:04.106418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677738a-eb34-42b4-9ea1-1b472aa5c03b None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=6ab2f376c7f043d5a8210e0e3fff03cb, user.id=29ea11eb17ec43978396819fc420fcce) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:01:04.106952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677738a-eb34-42b4-9ea1-1b472aa5c03b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:01:04.109131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677738a-eb34-42b4-9ea1-1b472aa5c03b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:01:04.115998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 608/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:01:03 2022] GET /identity/v3/role_assignments?user.id=29ea11eb17ec43978396819fc420fcce&scope.project.id=6ab2f376c7f043d5a8210e0e3fff03cb&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:01:13.959533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=31 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:14.113268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:01:59.678054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc84f-1297-4c83-982b-64132440c2e9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:59.678628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc84f-1297-4c83-982b-64132440c2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:59.678628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc84f-1297-4c83-982b-64132440c2e9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:59.680331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 593/1201] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:01:59.686244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e464a76-58db-4684-a30c-194d10e2d9ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:01:59.686706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e464a76-58db-4684-a30c-194d10e2d9ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:01:59.686793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e464a76-58db-4684-a30c-194d10e2d9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:01:59.710926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4e464a76-58db-4684-a30c-194d10e2d9ba None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:01:59.712326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e464a76-58db-4684-a30c-194d10e2d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:01:59.796388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 609/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:01:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:00.092103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcc84f-1297-4c83-982b-64132440c2e9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:00.217008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcc84f-1297-4c83-982b-64132440c2e9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:00.262591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:00.263248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:00.263529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:00.263828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:00.280816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:00.282357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:00.301647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:00.302956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:00.303973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:00.303973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:00.303973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:00.303973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 594/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:00.311452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e464a76-58db-4684-a30c-194d10e2d9ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:00.443671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e464a76-58db-4684-a30c-194d10e2d9ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:00.484350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:00.493876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a2cd6-683b-429b-84d5-596027be9b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:00.494748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2cd6-683b-429b-84d5-596027be9b0f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:00.495411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2cd6-683b-429b-84d5-596027be9b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:00.495739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2cd6-683b-429b-84d5-596027be9b0f None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:00.496576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2cd6-683b-429b-84d5-596027be9b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:00.497085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2cd6-683b-429b-84d5-596027be9b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:00.499091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2cd6-683b-429b-84d5-596027be9b0f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:00.523988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 610/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:00.524217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:00.593945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:00.660556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970a7a3-a696-40b3-bdf8-7efb6760cab5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:00.706884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c416099-d5d3-47d2-bb50-4daa04ee484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:00.707497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c416099-d5d3-47d2-bb50-4daa04ee484f None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:00.707855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c416099-d5d3-47d2-bb50-4daa04ee484f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:00.708144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c416099-d5d3-47d2-bb50-4daa04ee484f None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:00.708806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c416099-d5d3-47d2-bb50-4daa04ee484f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:00.710352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c416099-d5d3-47d2-bb50-4daa04ee484f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:00.746562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 595/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 6 01:02:00 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:02:02.983794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72b85-5a39-4dfc-a188-cfcf9e6d9c9c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:02.984341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72b85-5a39-4dfc-a188-cfcf9e6d9c9c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:02.984341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72b85-5a39-4dfc-a188-cfcf9e6d9c9c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:02.985655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 611/1206] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:02.991383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd1edb-52d4-4cfe-af07-40c0b8210d27 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:02.991964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd1edb-52d4-4cfe-af07-40c0b8210d27 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:02.992065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd1edb-52d4-4cfe-af07-40c0b8210d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:03.014418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c9fd1edb-52d4-4cfe-af07-40c0b8210d27 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:03.015956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c9fd1edb-52d4-4cfe-af07-40c0b8210d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:03.107788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 596/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:03.389544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b72b85-5a39-4dfc-a188-cfcf9e6d9c9c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:03.524093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b72b85-5a39-4dfc-a188-cfcf9e6d9c9c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:03.573044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:03.573853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:03.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:03.574413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:03.593493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:02:03.600729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:03.601681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:03.602550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 612/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 01:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:03.608699 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd1edb-52d4-4cfe-af07-40c0b8210d27 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:03.655627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:03.735970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd1edb-52d4-4cfe-af07-40c0b8210d27 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:03.783844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bf560-3494-4119-9599-91673b581376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:03.784712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bf560-3494-4119-9599-91673b581376 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:03.785118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bf560-3494-4119-9599-91673b581376 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:03.785470 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bf560-3494-4119-9599-91673b581376 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:03.786181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bf560-3494-4119-9599-91673b581376 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:03.786487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bf560-3494-4119-9599-91673b581376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:03.787922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bf560-3494-4119-9599-91673b581376 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:03.799430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 597/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 01:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:03.805829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:03.865230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:03.935875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d37671-62cd-43b3-bf38-40068ef6a661 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:03.986002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:03.986707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:03.987101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:03.987371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:04.000691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:04.003837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:04.018602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:04.019589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:04.020587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:04.020587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:04.020587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:04.021804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 613/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:04.027928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bf560-3494-4119-9599-91673b581376 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:04.170618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bf560-3494-4119-9599-91673b581376 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:04.172921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:04.219802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:04.220415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:04.220695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:04.220968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:04.221612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:04.221916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:04.223335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:04.236583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 598/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:04.242144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:04.376217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfadf8-d28d-4d79-a096-2d248ad9d60d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:04.379196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:04.425205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:04.425848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:04.426126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:04.426397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:04.435866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:04.437613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:02:04.452723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:04.455935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:04.457093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 614/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:04.459801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:04.588879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bef878-4bb8-4093-8d60-4b00e6396681 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:04.636127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:04.636738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:04.637077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:04.637335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:04.638087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:04.638396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:04.639967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:04.660532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 599/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 6 01:02:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:04.666706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:04.787095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d636b-f23c-48b4-9555-8a12d9fe3063 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:04.828316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:04.828973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:04.829227 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:04.829523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:04.860530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:04.907807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:04.907807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:05.027801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 615/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:02:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 01:02:05.029485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:05.052759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:05.146253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eaed5f-3ff3-42a4-b438-63d253035a5e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:05.181383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:05.182872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:05.183270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:05.183682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:05.194843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:05.196612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:05.213544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:05.214733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:05.220279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:05.220279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:05.220279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:05.220279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 600/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:05.220589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:05.284509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:05.326924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479ce15-60a4-4f2a-9e86-47bde7a83085 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:05.363839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:05.364498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:05.364784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:05.365050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:05.365694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:05.365994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:05.367349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:05.379794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 616/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:05.385431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:05.474438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:05.511873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fc24a-87c9-41b1-9552-ce068c85f2c9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:05.552043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:05.552677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:05.552954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:05.553311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:05.561985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:05.563483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:02:05.576906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:05.577872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:05.578794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:05.584153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:05.584322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 601/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:05.712245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325749a-3174-419c-9b52-dbea58362f02 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:05.748254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:05.764964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccfbed-84d4-489c-a35f-ce4cd921c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:05.765576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccfbed-84d4-489c-a35f-ce4cd921c89a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:05.765858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccfbed-84d4-489c-a35f-ce4cd921c89a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:05.766130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccfbed-84d4-489c-a35f-ce4cd921c89a None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:05.766851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccfbed-84d4-489c-a35f-ce4cd921c89a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:05.767156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccfbed-84d4-489c-a35f-ce4cd921c89a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:05.768606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccfbed-84d4-489c-a35f-ce4cd921c89a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:05.781387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 617/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:02:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:05.787441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:05.893555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b2991-b98b-4357-bd9d-8fe5b65ef3e2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:05.924758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:05.931635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5715f-f532-4653-931f-781651e83769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:05.932283 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5715f-f532-4653-931f-781651e83769 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:05.932565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5715f-f532-4653-931f-781651e83769 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:05.932840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5715f-f532-4653-931f-781651e83769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:05.933612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5715f-f532-4653-931f-781651e83769 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=3628a46d8f894d5481112ad00479a639) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:05.933880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5715f-f532-4653-931f-781651e83769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:05.935318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5715f-f532-4653-931f-781651e83769 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:05.941492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 602/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:02:05 2022] GET /identity/v3/role_assignments?user.id=3628a46d8f894d5481112ad00479a639&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:07.986655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440807-b987-45e0-98ee-087898d37401 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:07.987230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440807-b987-45e0-98ee-087898d37401 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:07.987230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440807-b987-45e0-98ee-087898d37401 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:07.988715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 618/1220] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:07.994878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7363e-7cf2-41e6-b844-a449dede10ce None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:07.995267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7363e-7cf2-41e6-b844-a449dede10ce None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:07.995433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7363e-7cf2-41e6-b844-a449dede10ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:08.017698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8dd7363e-7cf2-41e6-b844-a449dede10ce None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:08.019194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8dd7363e-7cf2-41e6-b844-a449dede10ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:08.095867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 603/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:08.384721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71440807-b987-45e0-98ee-087898d37401 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:08.489007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71440807-b987-45e0-98ee-087898d37401 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:08.527322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:08.528246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:08.528539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:08.528834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:08.539589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:02:08.545119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:08.546246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 01:02:08.547338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:08.551234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 619/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 01:02:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:08.552275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7363e-7cf2-41e6-b844-a449dede10ce None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:08.660651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7363e-7cf2-41e6-b844-a449dede10ce None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:08.686124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:08.697009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:08.697616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:08.697895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:08.698163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:08.698900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:08.699227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:08.700809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:08.709506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 604/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 01:02:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:08.715895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:08.763946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:08.825540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1224f-1b5c-47d5-b2a3-a9e575d841e0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:08.866168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:08.866908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:08.867200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:08.867541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:08.878226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:08.879963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:08.895911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:08.897024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:08.897949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:08.897949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:08.897949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:08.897949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 620/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:08.904213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:09.014627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df61ae-4507-42ef-aed8-e20fe76f70fb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:09.057542 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:09.058301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:09.058552 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:09.058936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:09.059683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:09.060040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:09.061767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:09.075162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 605/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:09.081208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:09.184995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:09.195488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d2c0f-b05f-4c79-92e3-76039f13aed0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:09.235246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:09.235909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:09.236193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:09.236527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:09.244622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:09.246286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:02:09.258766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:09.260461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:09.261402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 621/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:09.266710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:09.318443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:09.377771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e246a-ab38-4c0d-874a-23503635c5b6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:09.418682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:09.419281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:09.419559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:09.419854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:09.420610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:09.420911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:09.422492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:09.441955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 606/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 6 01:02:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:09.448735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:09.562486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3cfd0-d2f3-49b0-a9cd-b2393b8e735f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:09.602322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:09.602954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:09.603363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:09.603547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:09.618281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:09.620443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:09.636484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:09.637562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:09.638537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:09.638537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:09.638537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:09.638537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 622/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:09.643465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:09.744826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddcde1-f5b3-409a-99df-d900b36003f0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:09.747670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:09.783251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:09.783992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:09.784303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:09.784603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:09.785318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:09.785700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:09.787412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:09.803798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 607/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:02:09.807407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:09.817657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:09.933428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7a0a3-7583-430e-805c-c264db5a95fe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:09.970650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:09.971275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:09.971552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:09.971850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:09.979244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:09.980698 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:02:09.992552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:09.993541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:10.000395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 623/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:10.000906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:10.115722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c726c-d2d5-4506-b303-bde7b73d8b24 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:10.152854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcc4a6-fa74-4a5c-b824-158f9c6f728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:10.153460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcc4a6-fa74-4a5c-b824-158f9c6f728f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:10.153737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcc4a6-fa74-4a5c-b824-158f9c6f728f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:10.154007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcc4a6-fa74-4a5c-b824-158f9c6f728f None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:10.154693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfcc4a6-fa74-4a5c-b824-158f9c6f728f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:10.154996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfcc4a6-fa74-4a5c-b824-158f9c6f728f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:10.156455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfcc4a6-fa74-4a5c-b824-158f9c6f728f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:10.169602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 608/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:10.176954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:10.249904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:10.285097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a74be-a52a-4238-b07f-26df2331d31d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:10.320530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d4679-f10b-405e-8468-dc4eba2c4724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:10.321233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d4679-f10b-405e-8468-dc4eba2c4724 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:10.321569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d4679-f10b-405e-8468-dc4eba2c4724 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:10.321898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d4679-f10b-405e-8468-dc4eba2c4724 None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/3628a46d8f894d5481112ad00479a639/roles/b26b251c03cd4a2eac5f85927c38b797` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:10.340744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d4679-f10b-405e-8468-dc4eba2c4724 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=3628a46d8f894d5481112ad00479a639, role_id=b26b251c03cd4a2eac5f85927c38b797)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:10.342853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d4679-f10b-405e-8468-dc4eba2c4724 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:10.357577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 624/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 01:02:10 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/3628a46d8f894d5481112ad00479a639/roles/b26b251c03cd4a2eac5f85927c38b797 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:02:12.258306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33173676-d4e7-45c0-a429-841579c7f022 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:12.258865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33173676-d4e7-45c0-a429-841579c7f022 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:12.258977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33173676-d4e7-45c0-a429-841579c7f022 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:12.261073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 609/1233] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:12.266090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc8fd6-e36c-4bc1-bd6c-2b88695495d4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:12.266531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc8fd6-e36c-4bc1-bd6c-2b88695495d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:12.266621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc8fd6-e36c-4bc1-bd6c-2b88695495d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:12.286400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cdcc8fd6-e36c-4bc1-bd6c-2b88695495d4 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:12.287655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cdcc8fd6-e36c-4bc1-bd6c-2b88695495d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:12.362094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 625/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:12.628646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33173676-d4e7-45c0-a429-841579c7f022 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:12.655039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:12.740964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33173676-d4e7-45c0-a429-841579c7f022 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:12.780924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:12.781603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:12.781947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:12.782273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:12.793895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:02:12.798911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:12.800173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:12.801014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 610/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 6 01:02:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:12.807037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc8fd6-e36c-4bc1-bd6c-2b88695495d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:12.867673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:12.923088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc8fd6-e36c-4bc1-bd6c-2b88695495d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:12.961709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:12.962353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:12.962668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:12.963024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:12.963939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:12.964337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:12.965824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:12.974780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 626/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 6 01:02:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:12.981031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:13.083119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:13.100077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f569244-0e18-468c-bf43-8776c8c6be09 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:13.140606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:13.141143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:13.141424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:13.141696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:13.153075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:13.154586 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:13.172835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:13.174031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:13.175199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:13.175199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:13.175199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:13.175199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 611/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:13.180260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:13.281064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c2dd-3208-47de-b9a5-d4074de20f93 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:13.290417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:13.318272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:13.318880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:13.319159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:13.319432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:13.320104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:13.320459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:13.322476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:13.335189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 627/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:13.341300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:13.455466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa968f-eac8-4a90-a8e0-c78ccffe2c13 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:13.489584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:13.490207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:13.490486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:13.490750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:13.494187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:13.499884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:13.501585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:02:13.513264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:13.514253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:13.515215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 612/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:13.520813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:13.628977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4d19f-bc21-4a14-b979-5b20b1497924 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:13.664544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:13.665133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:13.665970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:13.666445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:13.667088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:13.667386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:13.668831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:13.686538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 628/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 6 01:02:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:02:13.692440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:13.804683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445df87f-b32f-4088-8435-f96f2c96688c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:13.844775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:13.845491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:13.845834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:13.846169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:13.909361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:13.917863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:13.919481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:14.049089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 613/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:02:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:02:14.050286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:14.080531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:14.156826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e520b7a-4d30-4184-b688-dbd13f22bc51 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:14.191842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:14.192471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:14.192752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:14.193026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:14.203023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:14.204495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:14.221535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:14.222651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:14.223560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:14.223560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:14.223560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:14.223560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 629/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:14.229249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:14.325290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:14.332087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fcb8c-bc80-4923-a82b-e84ee627ddf7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:14.374397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:14.375174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:14.375464 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:14.375943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:14.377095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:14.377632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:14.379989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:14.397572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 614/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:14.405062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:14.490049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:14.514247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834e3b8-edd7-478c-9f12-cb369bdf94e9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:14.550877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:14.551673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:14.552015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:14.552356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:14.559291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:14.560755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:02:14.574139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:14.575255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:14.576275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:14.580805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 630/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:02:14.583126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:14.693362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9bd08-4896-4338-be24-667be1ddf20f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:14.730578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ccbf9-3ccd-436e-8e51-d4c1bc579b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:14.731218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ccbf9-3ccd-436e-8e51-d4c1bc579b75 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:14.731500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ccbf9-3ccd-436e-8e51-d4c1bc579b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:14.731794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ccbf9-3ccd-436e-8e51-d4c1bc579b75 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:14.732548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ccbf9-3ccd-436e-8e51-d4c1bc579b75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:14.732851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ccbf9-3ccd-436e-8e51-d4c1bc579b75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:14.734495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ccbf9-3ccd-436e-8e51-d4c1bc579b75 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:14.747382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 615/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:02:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:02:14.754310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:14.860221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628bc0c-52d0-41a0-8ef0-9dbfcfc2db42 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:14.896528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083ccb3-b8d0-4070-b3aa-77da3f3a7102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:14.897155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083ccb3-b8d0-4070-b3aa-77da3f3a7102 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:14.897435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083ccb3-b8d0-4070-b3aa-77da3f3a7102 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:14.897710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083ccb3-b8d0-4070-b3aa-77da3f3a7102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:14.898436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083ccb3-b8d0-4070-b3aa-77da3f3a7102 None admin] RBAC: Adding query filter params (role.id=b26b251c03cd4a2eac5f85927c38b797, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=3628a46d8f894d5481112ad00479a639) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:14.898783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083ccb3-b8d0-4070-b3aa-77da3f3a7102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:14.900470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083ccb3-b8d0-4070-b3aa-77da3f3a7102 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:14.906621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 631/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:02:14 2022] GET /identity/v3/role_assignments?user.id=3628a46d8f894d5481112ad00479a639&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=b26b251c03cd4a2eac5f85927c38b797 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:17.099713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595e0ba-6c9d-4e3d-bfed-a342c6171bae None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:17.100273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595e0ba-6c9d-4e3d-bfed-a342c6171bae None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:17.100273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595e0ba-6c9d-4e3d-bfed-a342c6171bae None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:17.101753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 616/1247] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:17.107419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55eb55-dc48-49b3-ab3b-d587971f6743 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:17.107932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55eb55-dc48-49b3-ab3b-d587971f6743 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:17.108019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55eb55-dc48-49b3-ab3b-d587971f6743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:17.114555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:17.127966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ab55eb55-dc48-49b3-ab3b-d587971f6743 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:17.129193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ab55eb55-dc48-49b3-ab3b-d587971f6743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:17.202940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 632/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:17.498892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595e0ba-6c9d-4e3d-bfed-a342c6171bae None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:17.518487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:17.615209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595e0ba-6c9d-4e3d-bfed-a342c6171bae None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:17.654491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:17.655131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:17.655416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:17.655826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:17.669789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:02:17.675010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:17.676232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:17.677320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 617/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:17.682273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55eb55-dc48-49b3-ab3b-d587971f6743 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:17.801258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55eb55-dc48-49b3-ab3b-d587971f6743 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:17.834475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:17.840987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:17.841662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:17.841970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:17.842273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:17.842989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:17.843316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:17.844833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:17.855555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 633/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:17.860416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:17.942835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:17.971170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dbe49-53a0-4d1f-b9ec-12f3c93bed85 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:18.010397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:18.011178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:18.011722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:18.012119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:18.023679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:18.026218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:18.047525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:18.048894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:18.050164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:18.050164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:18.050164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:18.050164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 618/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:18.055357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:18.176768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba02e03-4809-4fcc-b83b-e626fdc565ae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:18.210659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:18.211279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:18.211548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:18.211853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:18.213328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:18.213897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:18.216369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:18.230474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 634/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:18.236405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:18.346123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ac1d-af55-411c-adc6-df6adb7a9099 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:18.376354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:18.385457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:18.386087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:18.386365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:18.386637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:18.394766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:18.396276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:02:18.407975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:18.409072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:18.410054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 619/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:18.415040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:18.477409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:18.525697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ebc68-c498-4183-be6c-c1535c95f29a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:18.562800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:18.563543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:18.563875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:18.564185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:18.565001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:18.565377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:18.566960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:18.589752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 635/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 6 01:02:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:02:18.596001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:18.704696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab960c-842f-421c-b2f1-0a9efb028e2f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:18.739769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:18.740433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:18.740716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:18.740988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:18.802658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:18.817561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:18.819126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:18.922753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 620/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:02:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:02:18.928896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:19.036846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afae4a-5e28-4091-9551-787cc959c5a0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:19.054177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:19.079805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:19.080928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:19.081408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:19.081888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:19.094927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:19.096505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:19.113176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:19.114522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:19.115714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:19.115714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:19.115714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:19.115714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 636/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:19.120830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:19.214724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:19.229098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debca031-8930-40c6-8ae3-6d68ad7553a9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:19.265495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:19.266228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:19.266365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:19.266636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:19.267278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:19.267572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:19.268981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:19.281766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 621/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:19.287737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:19.399283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0ff29-b7d4-4d8c-8783-ec0a751b543e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:19.436342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:19.436979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:19.437262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:19.437535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:19.444940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:19.446481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:02:19.458695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:19.459906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:19.461056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 637/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:02:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:02:19.466750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:19.573715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83ef44-c1ba-4538-a20f-adf082b20216 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:19.611581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc809ce-b97e-461a-88c1-cc9298da73ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:19.612228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc809ce-b97e-461a-88c1-cc9298da73ec None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:19.612509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc809ce-b97e-461a-88c1-cc9298da73ec None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:19.612781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc809ce-b97e-461a-88c1-cc9298da73ec None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:19.613477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc809ce-b97e-461a-88c1-cc9298da73ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:19.613780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc809ce-b97e-461a-88c1-cc9298da73ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:19.615226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc809ce-b97e-461a-88c1-cc9298da73ec None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:19.628195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 622/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:02:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:19.634972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:19.663135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:19.741801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ebf96-fe7c-4f7f-9a49-9e8d81301835 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:19.780009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a78e9-8e28-45f7-b4eb-f5752431b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:19.781281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a78e9-8e28-45f7-b4eb-f5752431b941 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:19.781572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a78e9-8e28-45f7-b4eb-f5752431b941 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:19.781849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a78e9-8e28-45f7-b4eb-f5752431b941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:19.782587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a78e9-8e28-45f7-b4eb-f5752431b941 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=3628a46d8f894d5481112ad00479a639) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:19.782903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a78e9-8e28-45f7-b4eb-f5752431b941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:19.784379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a78e9-8e28-45f7-b4eb-f5752431b941 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:19.792296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 638/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:02:19 2022] GET /identity/v3/role_assignments?user.id=3628a46d8f894d5481112ad00479a639&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:21.656163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71b5f-d272-4c03-b358-0aa72739aa0c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:21.656654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71b5f-d272-4c03-b358-0aa72739aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:21.656739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71b5f-d272-4c03-b358-0aa72739aa0c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:21.658055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 623/1261] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:21.662996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3709ef-0ae7-40d1-ac7d-60b0f6eb6549 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:21.663427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3709ef-0ae7-40d1-ac7d-60b0f6eb6549 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:21.663544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3709ef-0ae7-40d1-ac7d-60b0f6eb6549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:21.682641 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0d3709ef-0ae7-40d1-ac7d-60b0f6eb6549 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:21.683851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0d3709ef-0ae7-40d1-ac7d-60b0f6eb6549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:21.747875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 639/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:22.017447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c71b5f-d272-4c03-b358-0aa72739aa0c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:22.028462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:22.123944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c71b5f-d272-4c03-b358-0aa72739aa0c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:22.155969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:22.156583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:22.156859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:22.157128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:22.167566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:02:22.172306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:22.173225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:22.174064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 624/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:22.179497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3709ef-0ae7-40d1-ac7d-60b0f6eb6549 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:22.197099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:22.276461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3709ef-0ae7-40d1-ac7d-60b0f6eb6549 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:22.309203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:22.309802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:22.310083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:22.310356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:22.311004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:22.311305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:22.312695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:22.320368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 640/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:02:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:22.326224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:22.390352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:22.423891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778174f-8bc6-4553-9fde-22a413996d77 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:22.455526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:22.456164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:22.456441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:22.456708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:22.466754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:22.468223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:22.482670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:22.483822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:22.484746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:22.484746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:22.484746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:22.484746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 625/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:22.489817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:22.569744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:22.590386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043cb77-27db-4a86-8ed1-018da46647dc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:22.623193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:22.623810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:22.624096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:22.624374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:22.625012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:22.625316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:22.626665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:22.638403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 641/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:22.644168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:22.739091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3563da-9688-4f7c-8092-b6b09c9bfed3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:22.757647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:22.772207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:22.772828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:22.773107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:22.773376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:22.781206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:22.782632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:02:22.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:22.794780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:22.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 626/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:22.800546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:22.901025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1b6a4-8d7b-41ca-8e39-f3196698e58e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:22.934640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:22.935245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:22.935528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:22.935832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:22.936564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:22.936872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:22.938281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:22.949193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:22.957848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 642/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 6 01:02:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:22.963570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:23.077146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c0a96-61a5-4521-b071-dca52a87d34d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:23.109063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:23.109685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:23.109965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:23.110236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:23.120876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:23.122301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:23.136490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:23.137598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:23.138545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:23.138545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:23.138545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:23.138545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 627/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:23.143423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:23.237751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdcffc-162e-4c5d-8870-a4dacd9ad5f2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:23.270793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:23.271377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:23.271654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:23.271950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:23.272579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:23.272877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:23.274219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:23.284918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 643/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:23.290802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:23.321672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:23.387624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b0a20-9d30-4d30-9649-406e3d79b626 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:23.420169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:23.420774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:23.421052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:23.421319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:23.427637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:23.429101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:02:23.440523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:23.441553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:23.442461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 628/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:02:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:23.447187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:23.504965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:23.546074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4650bb2-7fe1-4af5-8eb1-737f2d3e1ac3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:23.579924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbbea2-5ad7-456e-b877-f6d7e977426b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:23.580524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbbea2-5ad7-456e-b877-f6d7e977426b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:23.581050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbbea2-5ad7-456e-b877-f6d7e977426b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:23.581548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbbea2-5ad7-456e-b877-f6d7e977426b None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:23.583594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbbea2-5ad7-456e-b877-f6d7e977426b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:23.583594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbbea2-5ad7-456e-b877-f6d7e977426b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:23.584497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbbea2-5ad7-456e-b877-f6d7e977426b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:23.598738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 644/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:02:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:02:23.605303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:23.693171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:23.701836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8dbe5-7ab5-4a36-bae2-78ad36d0666f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:23.734248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85008920-afe7-4c44-85fa-dca0f9ea0ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:23.734887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85008920-afe7-4c44-85fa-dca0f9ea0ad3 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:23.735167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85008920-afe7-4c44-85fa-dca0f9ea0ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:23.735441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85008920-afe7-4c44-85fa-dca0f9ea0ad3 None admin] PATH_INFO: `/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/3628a46d8f894d5481112ad00479a639/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:23.737327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85008920-afe7-4c44-85fa-dca0f9ea0ad3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e0c168c2f1476599951c75ac91e4e7, user_id=3628a46d8f894d5481112ad00479a639, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:23.738755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85008920-afe7-4c44-85fa-dca0f9ea0ad3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:23.750935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 629/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 6 01:02:23 2022] PUT /identity/v3/projects/a1e0c168c2f1476599951c75ac91e4e7/users/3628a46d8f894d5481112ad00479a639/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:02:25.612223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9844b21-3b32-4063-af59-3703f070d99d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:25.612766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9844b21-3b32-4063-af59-3703f070d99d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:25.612766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9844b21-3b32-4063-af59-3703f070d99d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:25.614118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 645/1274] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:25.619317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284537c-326f-482c-be96-50fabc4eccbd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:25.619720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284537c-326f-482c-be96-50fabc4eccbd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:25.619902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284537c-326f-482c-be96-50fabc4eccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:25.638067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d284537c-326f-482c-be96-50fabc4eccbd None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:25.639252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d284537c-326f-482c-be96-50fabc4eccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:25.703202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 630/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:25.968355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9844b21-3b32-4063-af59-3703f070d99d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:26.072823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9844b21-3b32-4063-af59-3703f070d99d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:26.089398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:26.105307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:26.105919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:26.106193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:26.106466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:26.115974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 01:02:26.121166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:26.122243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:26.123165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 646/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 6 01:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:26.128264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284537c-326f-482c-be96-50fabc4eccbd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:26.206719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:26.225391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284537c-326f-482c-be96-50fabc4eccbd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:26.257706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:26.258301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:26.258583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:26.259394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:26.260075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:26.260379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:26.261739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:26.269383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 631/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 6 01:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:26.274891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:26.369859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda02ee-21d9-42df-9ca8-b9ba030b7880 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:26.403636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:26.404272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:26.404556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:26.404828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:26.414339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:26.416562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:26.430401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:26.432086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:26.433169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:26.433169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:26.433169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:26.433169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 647/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:26.437816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:26.533072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60709ae-92b5-48c2-8808-3df4b6957f92 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:26.565728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f498-4de3-4894-a330-87535e491776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:26.566315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f498-4de3-4894-a330-87535e491776 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:26.566594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f498-4de3-4894-a330-87535e491776 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:26.566865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f498-4de3-4894-a330-87535e491776 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:26.567490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120f498-4de3-4894-a330-87535e491776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:26.567812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120f498-4de3-4894-a330-87535e491776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:26.569196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120f498-4de3-4894-a330-87535e491776 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:26.580556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 632/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:26.586072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:26.615818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:26.682183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bba395-bc1d-4cba-982b-8e62930e1568 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:26.713769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:26.714371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:26.714648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:26.714914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:26.721872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:26.723273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 01:02:26.734427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:26.735535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:26.736539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 648/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:26.741096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f498-4de3-4894-a330-87535e491776 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:26.744857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:26.836693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f498-4de3-4894-a330-87535e491776 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:26.869599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:26.870195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:26.870470 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:26.870738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:26.871451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:26.871781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:26.873192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:26.889443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 633/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 6 01:02:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:26.895235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:26.982289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:26.992121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c0bf0-3c84-44c6-b232-3d3ab1a8c472 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:27.029138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:27.029745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:27.030140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:27.030280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:27.095826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:27.097216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:27.181047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:27.191410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 649/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 6 01:02:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 01:02:27.197623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:27.256800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:27.294712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3373-9460-4dec-ad23-439d1253d51a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:27.326974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:27.327590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:27.327900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:27.328181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:27.338440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:27.339901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 01:02:27.353878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:27.354987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 01:02:27.356147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 01:02:27.356147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 01:02:27.356147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:27.356147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 634/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 01:02:27.360542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:27.455550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4e3f1-cb5d-47f4-8cd5-9fb928a0a789 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:27.487448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:27.488075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:27.488356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:27.488627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:27.489252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:27.489552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:27.490881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:27.502037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 650/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 6 01:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:27.507358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:27.602714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963c92d-7b9c-4428-ac18-8ccbd60b2b95 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:27.615570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:27.636028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:27.636638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:27.636914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:27.637181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:27.643434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:27.644900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 01:02:27.655787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:27.656832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 06 01:02:27.657921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 635/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 6 01:02:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:27.662783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:27.711586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:27.759994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7c0f9-dfa8-4607-98d5-634725c2fe14 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:27.792199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c4e94-6e21-40a6-a832-bff209ad6790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:27.792799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4e94-6e21-40a6-a832-bff209ad6790 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:27.793081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4e94-6e21-40a6-a832-bff209ad6790 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:27.793355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4e94-6e21-40a6-a832-bff209ad6790 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:27.794040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c4e94-6e21-40a6-a832-bff209ad6790 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:27.794348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c4e94-6e21-40a6-a832-bff209ad6790 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:27.795773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c4e94-6e21-40a6-a832-bff209ad6790 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:27.809291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 651/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 6 01:02:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:27.815397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:27.913967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877c620-2324-46f9-8e31-a26cb9c4e03a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:27.947313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56855ea-06e9-4601-aa1c-021ce85dc9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:27.947927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56855ea-06e9-4601-aa1c-021ce85dc9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:27.948199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56855ea-06e9-4601-aa1c-021ce85dc9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:27.948463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56855ea-06e9-4601-aa1c-021ce85dc9a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:27.949173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56855ea-06e9-4601-aa1c-021ce85dc9a8 None admin] RBAC: Adding query filter params (role.id=40b543bb076c4ea9a663ec990f6582b6, scope.project.id=a1e0c168c2f1476599951c75ac91e4e7, user.id=3628a46d8f894d5481112ad00479a639) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:27.949472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56855ea-06e9-4601-aa1c-021ce85dc9a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:27.950896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56855ea-06e9-4601-aa1c-021ce85dc9a8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:27.956584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 636/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 6 01:02:27 2022] GET /identity/v3/role_assignments?user.id=3628a46d8f894d5481112ad00479a639&scope.project.id=a1e0c168c2f1476599951c75ac91e4e7&role.id=40b543bb076c4ea9a663ec990f6582b6 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:29.857567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943b621-c768-42fb-b949-5414d4020be9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:29.858105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943b621-c768-42fb-b949-5414d4020be9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:29.858105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943b621-c768-42fb-b949-5414d4020be9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:29.859342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 652/1288] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:29.864350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac57fc-627c-4089-b22c-6a2823764788 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:29.864768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac57fc-627c-4089-b22c-6a2823764788 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:29.864895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac57fc-627c-4089-b22c-6a2823764788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:29.884050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-75ac57fc-627c-4089-b22c-6a2823764788 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:29.885523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-75ac57fc-627c-4089-b22c-6a2823764788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:29.914264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:29.950648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 637/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:30.227867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943b621-c768-42fb-b949-5414d4020be9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:30.325436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943b621-c768-42fb-b949-5414d4020be9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:30.335585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:30.358795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:30.359407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:30.359687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:30.359990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:30.360563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:30.361922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:30.370507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:30.371871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:30.373103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 653/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 01:02:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:30.377386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac57fc-627c-4089-b22c-6a2823764788 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:30.473437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac57fc-627c-4089-b22c-6a2823764788 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:30.505311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d803b7-1c1c-42fc-9e08-4cdc3cb1a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:30.505845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d803b7-1c1c-42fc-9e08-4cdc3cb1a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:30.506120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d803b7-1c1c-42fc-9e08-4cdc3cb1a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:30.506388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d803b7-1c1c-42fc-9e08-4cdc3cb1a2b5 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:30.507001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d803b7-1c1c-42fc-9e08-4cdc3cb1a2b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:30.507297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d803b7-1c1c-42fc-9e08-4cdc3cb1a2b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:30.509255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d803b7-1c1c-42fc-9e08-4cdc3cb1a2b5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:30.513817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 638/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 6 01:02:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:30.519412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:30.615533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1bbd1-b58e-4db0-abef-1f19f49ebe0d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:30.647769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e725b3-6d19-45df-816d-72b035f9d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:30.648372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e725b3-6d19-45df-816d-72b035f9d24f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:30.648650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e725b3-6d19-45df-816d-72b035f9d24f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:30.648918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e725b3-6d19-45df-816d-72b035f9d24f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:30.649556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e725b3-6d19-45df-816d-72b035f9d24f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:30.649852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e725b3-6d19-45df-816d-72b035f9d24f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:30.651189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e725b3-6d19-45df-816d-72b035f9d24f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:30.655388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 654/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 6 01:02:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:32.552692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15165a-ac3a-4fe0-89f2-3519b8ed4d91 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:32.553258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15165a-ac3a-4fe0-89f2-3519b8ed4d91 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:32.553258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15165a-ac3a-4fe0-89f2-3519b8ed4d91 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:32.554548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 639/1293] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:32.559890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb865bdb-6976-49fe-bdba-2909d768ba84 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:32.560275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb865bdb-6976-49fe-bdba-2909d768ba84 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:32.560452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb865bdb-6976-49fe-bdba-2909d768ba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:32.578825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bb865bdb-6976-49fe-bdba-2909d768ba84 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:32.580047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb865bdb-6976-49fe-bdba-2909d768ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:32.609133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:32.645148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 655/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 01:02:32.915149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15165a-ac3a-4fe0-89f2-3519b8ed4d91 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:32.945488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:33.016496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15165a-ac3a-4fe0-89f2-3519b8ed4d91 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:33.060346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef1f5b-1904-4a8a-8c40-ced780debe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:33.061063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef1f5b-1904-4a8a-8c40-ced780debe7c None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:33.061421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef1f5b-1904-4a8a-8c40-ced780debe7c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:33.061765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef1f5b-1904-4a8a-8c40-ced780debe7c None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:33.062558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef1f5b-1904-4a8a-8c40-ced780debe7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:33.064274 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef1f5b-1904-4a8a-8c40-ced780debe7c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:33.080116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 640/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 6 01:02:32 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:02:34.981687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f2274-51b6-464e-9238-55354542c26b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:34.982214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f2274-51b6-464e-9238-55354542c26b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:34.982318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f2274-51b6-464e-9238-55354542c26b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:34.983570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 656/1296] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:34.988631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da623823-bb5f-4572-8992-de6f8074a6bb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:34.989064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da623823-bb5f-4572-8992-de6f8074a6bb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:34.989163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da623823-bb5f-4572-8992-de6f8074a6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:35.007385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-da623823-bb5f-4572-8992-de6f8074a6bb None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:35.008605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-da623823-bb5f-4572-8992-de6f8074a6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:35.079596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 641/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:35.347918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f2274-51b6-464e-9238-55354542c26b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:35.443842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f2274-51b6-464e-9238-55354542c26b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:35.476426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:35.477050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:35.477327 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:35.477598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:35.478168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:35.479514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:35.487897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:35.489063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:35.490146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 657/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 01:02:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:35.494804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da623823-bb5f-4572-8992-de6f8074a6bb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:35.512911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:35.591658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da623823-bb5f-4572-8992-de6f8074a6bb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:35.623318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:35.623880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:35.624159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:35.624430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:35.625048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:35.625344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:35.626701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:35.631704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 642/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 6 01:02:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:35.638153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:35.668886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:35.734225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2156f-ae33-4414-8ecf-fc17c6631125 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:35.766470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492b1db-c131-45af-8c8e-61f0f7ee5a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:35.767044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492b1db-c131-45af-8c8e-61f0f7ee5a62 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:35.767320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492b1db-c131-45af-8c8e-61f0f7ee5a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:35.767587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492b1db-c131-45af-8c8e-61f0f7ee5a62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:35.768294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492b1db-c131-45af-8c8e-61f0f7ee5a62 None admin] RBAC: Adding query filter params (interface=public, service_id=455183d054d049619710fbb27bdf8452, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:35.768599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492b1db-c131-45af-8c8e-61f0f7ee5a62 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:35.769934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492b1db-c131-45af-8c8e-61f0f7ee5a62 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:35.774696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 658/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 6 01:02:35 2022] GET /identity/v3/endpoints?service_id=455183d054d049619710fbb27bdf8452&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:35.780127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:35.857602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:35.875733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989031b-990d-4606-908f-1a8e3667526b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:35.908481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336b6f0-a418-4e2c-9084-fca910e715d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:35.909066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336b6f0-a418-4e2c-9084-fca910e715d3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:35.909342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336b6f0-a418-4e2c-9084-fca910e715d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:35.909610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336b6f0-a418-4e2c-9084-fca910e715d3 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:35.910197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336b6f0-a418-4e2c-9084-fca910e715d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:35.911539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336b6f0-a418-4e2c-9084-fca910e715d3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:35.918282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 643/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 6 01:02:35 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:02:37.799214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee1206-a4ae-4fd0-a2fd-3a70b16a1ca1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:37.799773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee1206-a4ae-4fd0-a2fd-3a70b16a1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:37.799773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee1206-a4ae-4fd0-a2fd-3a70b16a1ca1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:37.801099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 659/1302] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:02:37.806733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c839f-05e4-4a07-bc99-89c11939cc87 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:37.807157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c839f-05e4-4a07-bc99-89c11939cc87 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:37.807282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c839f-05e4-4a07-bc99-89c11939cc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:37.824969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-367c839f-05e4-4a07-bc99-89c11939cc87 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:02:37.826144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-367c839f-05e4-4a07-bc99-89c11939cc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:02:37.889814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 644/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:02:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 01:02:38.176704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee1206-a4ae-4fd0-a2fd-3a70b16a1ca1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:38.268798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:38.276447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee1206-a4ae-4fd0-a2fd-3a70b16a1ca1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:38.308637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:38.309248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:38.309527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:38.309799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:38.310369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:38.311715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 01:02:38.320219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 01:02:38.321850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 06 01:02:38.323074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 660/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 6 01:02:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 01:02:38.327685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c839f-05e4-4a07-bc99-89c11939cc87 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:38.401181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:38.423726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c839f-05e4-4a07-bc99-89c11939cc87 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:38.458592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe343282-45da-49fe-8f49-4b529cf5c415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:38.459132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe343282-45da-49fe-8f49-4b529cf5c415 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:38.459411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe343282-45da-49fe-8f49-4b529cf5c415 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:38.459680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe343282-45da-49fe-8f49-4b529cf5c415 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:38.460400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe343282-45da-49fe-8f49-4b529cf5c415 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:02:38.460699 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe343282-45da-49fe-8f49-4b529cf5c415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:38.462726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe343282-45da-49fe-8f49-4b529cf5c415 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:38.467679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 645/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 6 01:02:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:02:38.473315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:02:38.569137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364644-84b4-4710-8c45-21680ee49ab2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:02:38.601981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27a096-c776-43b3-9f16-b51cb9a8daa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:02:38.602573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a096-c776-43b3-9f16-b51cb9a8daa9 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:02:38.602858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a096-c776-43b3-9f16-b51cb9a8daa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:02:38.603130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a096-c776-43b3-9f16-b51cb9a8daa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:02:38.603817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27a096-c776-43b3-9f16-b51cb9a8daa9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:02:38.605150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27a096-c776-43b3-9f16-b51cb9a8daa9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:02:38.623801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 661/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:02:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:02:48.465183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=30 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:02:48.619879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:41.587589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234705-845c-47ed-bf0c-3bdf293f644c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:41.588066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234705-845c-47ed-bf0c-3bdf293f644c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:41.588165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234705-845c-47ed-bf0c-3bdf293f644c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:41.592136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 646/1307] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:03:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:41.594544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31311fe-67d5-4084-b456-dd5aab96ba40 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:41.594993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31311fe-67d5-4084-b456-dd5aab96ba40 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:41.595077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31311fe-67d5-4084-b456-dd5aab96ba40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:41.683966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b31311fe-67d5-4084-b456-dd5aab96ba40 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:41.685308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b31311fe-67d5-4084-b456-dd5aab96ba40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:41.820517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 662/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:03:42.137966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d5a3e-b9b4-4fd1-820f-cb393379ad08 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:42.138402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d5a3e-b9b4-4fd1-820f-cb393379ad08 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:42.138566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d5a3e-b9b4-4fd1-820f-cb393379ad08 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:42.140153 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 647/1309] 10.222.0.23 () {54 vars in 965 bytes} [Tue Sep 6 01:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:42.146390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ddd69-2a42-46d5-b241-1437db6436c4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:42.146757 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ddd69-2a42-46d5-b241-1437db6436c4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:42.147112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ddd69-2a42-46d5-b241-1437db6436c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:42.225489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cc3ddd69-2a42-46d5-b241-1437db6436c4 None None] MFA Rules not processed for user `bc47f8d295ed481a87b4fb0d405c605e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:42.226687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cc3ddd69-2a42-46d5-b241-1437db6436c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:42.248125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:42.307787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 663/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Tue Sep 6 01:03:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:03:42.316771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d5a3e-b9b4-4fd1-820f-cb393379ad08 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:03:42.318642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f23d5a3e-b9b4-4fd1-820f-cb393379ad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:42.439601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d5a3e-b9b4-4fd1-820f-cb393379ad08 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:03:42.530020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fd25d-0ffa-4754-9157-cde1508f4254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:03:42.530644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fd25d-0ffa-4754-9157-cde1508f4254 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:42.530924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fd25d-0ffa-4754-9157-cde1508f4254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:42.531197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fd25d-0ffa-4754-9157-cde1508f4254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:42.532935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-587fd25d-0ffa-4754-9157-cde1508f4254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:42.592882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:42.653798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fd25d-0ffa-4754-9157-cde1508f4254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:03:42.655282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fd25d-0ffa-4754-9157-cde1508f4254 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:03:42.675929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 648/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:03:44.853330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb9e8f-fc6d-481a-bee8-97682925dd56 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:44.853870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb9e8f-fc6d-481a-bee8-97682925dd56 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:44.853870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb9e8f-fc6d-481a-bee8-97682925dd56 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:44.855129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 664/1312] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:44.860638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27da1f-24c9-4686-9225-5f47418d9c6d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:44.861070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27da1f-24c9-4686-9225-5f47418d9c6d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:44.861189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27da1f-24c9-4686-9225-5f47418d9c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:44.945091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fd27da1f-24c9-4686-9225-5f47418d9c6d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:44.946466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fd27da1f-24c9-4686-9225-5f47418d9c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:45.053902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 649/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:03:45.335781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb9e8f-fc6d-481a-bee8-97682925dd56 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:03:45.337423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-07eb9e8f-fc6d-481a-bee8-97682925dd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:45.441719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb9e8f-fc6d-481a-bee8-97682925dd56 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:03:45.511342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:45.514043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb31cf9-0e31-4d14-a8bd-5ff060890267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:03:45.514686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb31cf9-0e31-4d14-a8bd-5ff060890267 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:45.515140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb31cf9-0e31-4d14-a8bd-5ff060890267 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:45.515573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb31cf9-0e31-4d14-a8bd-5ff060890267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:45.517483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5bb31cf9-0e31-4d14-a8bd-5ff060890267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:45.627096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb31cf9-0e31-4d14-a8bd-5ff060890267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:03:45.628547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb31cf9-0e31-4d14-a8bd-5ff060890267 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:03:45.646909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 665/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:03:47.863523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a7f82-d183-4d5e-a13e-e3c43bb1fee2 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:47.864097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a7f82-d183-4d5e-a13e-e3c43bb1fee2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:47.864097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a7f82-d183-4d5e-a13e-e3c43bb1fee2 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:47.865889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 650/1315] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:47.871093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4958c-8a8d-4a5d-b429-4549e8767e48 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:47.871508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4958c-8a8d-4a5d-b429-4549e8767e48 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:47.871645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4958c-8a8d-4a5d-b429-4549e8767e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:47.936045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-01d4958c-8a8d-4a5d-b429-4549e8767e48 None None] MFA Rules not processed for user `47aae071f79647d0a619a13ffb7ae739`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:47.937239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-01d4958c-8a8d-4a5d-b429-4549e8767e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:47.992216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:48.038472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 666/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:03:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:03:48.326532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a7f82-d183-4d5e-a13e-e3c43bb1fee2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:03:48.328110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-042a7f82-d183-4d5e-a13e-e3c43bb1fee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:48.337820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:48.432877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a7f82-d183-4d5e-a13e-e3c43bb1fee2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:03:48.503612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fa15f-4028-4c1f-a1d5-9f6b8371c02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:03:48.504242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa15f-4028-4c1f-a1d5-9f6b8371c02d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:48.504520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa15f-4028-4c1f-a1d5-9f6b8371c02d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:48.504814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fa15f-4028-4c1f-a1d5-9f6b8371c02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:48.506417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-886fa15f-4028-4c1f-a1d5-9f6b8371c02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:48.568107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:48.618181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fa15f-4028-4c1f-a1d5-9f6b8371c02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:03:48.619620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fa15f-4028-4c1f-a1d5-9f6b8371c02d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:03:48.637893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 651/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:03:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:03:51.085561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cbf2a-b144-4cf4-9c11-7b4ce1b86f01 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:51.086098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cbf2a-b144-4cf4-9c11-7b4ce1b86f01 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:51.086098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cbf2a-b144-4cf4-9c11-7b4ce1b86f01 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:51.087468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 667/1318] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:51.092517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea922997-a1f3-4ff4-9120-50b354e9cf9b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:51.092893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea922997-a1f3-4ff4-9120-50b354e9cf9b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:51.093056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea922997-a1f3-4ff4-9120-50b354e9cf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:51.158169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ea922997-a1f3-4ff4-9120-50b354e9cf9b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:51.159343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ea922997-a1f3-4ff4-9120-50b354e9cf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:51.253791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 652/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:03:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:03:51.537584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cbf2a-b144-4cf4-9c11-7b4ce1b86f01 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:03:51.539145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-846cbf2a-b144-4cf4-9c11-7b4ce1b86f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:51.640984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cbf2a-b144-4cf4-9c11-7b4ce1b86f01 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:03:51.712764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8331b-683f-40cb-9c3c-e0fa15e652be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:03:51.713365 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8331b-683f-40cb-9c3c-e0fa15e652be service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:51.713646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8331b-683f-40cb-9c3c-e0fa15e652be service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:51.713921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8331b-683f-40cb-9c3c-e0fa15e652be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:51.715529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b3f8331b-683f-40cb-9c3c-e0fa15e652be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:51.722750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:51.821621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8331b-683f-40cb-9c3c-e0fa15e652be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:03:51.822991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8331b-683f-40cb-9c3c-e0fa15e652be service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:03:51.841307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 668/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:03:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:03:54.342625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a984-48a3-4248-a51d-3329f5efe676 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:54.343158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a984-48a3-4248-a51d-3329f5efe676 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:54.343158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5a984-48a3-4248-a51d-3329f5efe676 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:54.344578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 653/1321] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:54.350318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632843a-5476-41b0-9fef-2750baeccd17 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:54.350736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632843a-5476-41b0-9fef-2750baeccd17 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:54.350900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632843a-5476-41b0-9fef-2750baeccd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:54.416106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d632843a-5476-41b0-9fef-2750baeccd17 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:54.417309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d632843a-5476-41b0-9fef-2750baeccd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:54.474000 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:54.511547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 669/1322] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:03:54.790958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5a984-48a3-4248-a51d-3329f5efe676 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:03:54.792547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d2b5a984-48a3-4248-a51d-3329f5efe676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:54.812141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:54.900595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5a984-48a3-4248-a51d-3329f5efe676 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:03:54.972426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2ef62-6b1c-408a-9144-5597ff85cc06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:03:54.973026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2ef62-6b1c-408a-9144-5597ff85cc06 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:54.973821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2ef62-6b1c-408a-9144-5597ff85cc06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:54.974089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2ef62-6b1c-408a-9144-5597ff85cc06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:54.975676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5de2ef62-6b1c-408a-9144-5597ff85cc06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:55.045544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:55.083069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2ef62-6b1c-408a-9144-5597ff85cc06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:03:55.084534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2ef62-6b1c-408a-9144-5597ff85cc06 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:03:55.102710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 654/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:03:57.042313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45334c5-2081-4081-8858-f84063f6fc61 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:57.042834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45334c5-2081-4081-8858-f84063f6fc61 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:57.042946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45334c5-2081-4081-8858-f84063f6fc61 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:57.044247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 670/1324] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:57.049448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc8f4b-2c39-4fae-879c-cdfdb6ec03c3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:57.049823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc8f4b-2c39-4fae-879c-cdfdb6ec03c3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:57.049995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc8f4b-2c39-4fae-879c-cdfdb6ec03c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:57.116432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ccbc8f4b-2c39-4fae-879c-cdfdb6ec03c3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:57.117524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ccbc8f4b-2c39-4fae-879c-cdfdb6ec03c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:57.216077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 655/1325] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:03:57.508461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39370a0a-47f9-48f0-acc0-de64cfaf0af9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:57.508842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39370a0a-47f9-48f0-acc0-de64cfaf0af9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:57.509186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39370a0a-47f9-48f0-acc0-de64cfaf0af9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:57.510625 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 671/1326] 10.222.0.23 () {54 vars in 965 bytes} [Tue Sep 6 01:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:03:57.517312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b488597-60d4-4480-b93b-0437a8625594 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:57.517600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b488597-60d4-4480-b93b-0437a8625594 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:57.517874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b488597-60d4-4480-b93b-0437a8625594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:57.548432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:57.595817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6b488597-60d4-4480-b93b-0437a8625594 None None] MFA Rules not processed for user `bc47f8d295ed481a87b4fb0d405c605e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:03:57.597034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6b488597-60d4-4480-b93b-0437a8625594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:57.671854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 656/1327] 10.222.0.23 () {58 vars in 1062 bytes} [Tue Sep 6 01:03:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:03:57.681602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39370a0a-47f9-48f0-acc0-de64cfaf0af9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:03:57.683174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-39370a0a-47f9-48f0-acc0-de64cfaf0af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:57.797056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39370a0a-47f9-48f0-acc0-de64cfaf0af9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:03:57.870937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d831d-5d5e-4377-85ae-24c45ee23cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:03:57.871545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d831d-5d5e-4377-85ae-24c45ee23cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:03:57.871854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d831d-5d5e-4377-85ae-24c45ee23cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:03:57.872128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d831d-5d5e-4377-85ae-24c45ee23cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:03:57.873729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-632d831d-5d5e-4377-85ae-24c45ee23cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:03:57.890402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:03:57.980958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d831d-5d5e-4377-85ae-24c45ee23cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:03:57.982272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d831d-5d5e-4377-85ae-24c45ee23cf9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:03:58.000697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 672/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:00.178683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e0d7-b45f-4c22-82a7-e0f1c324b189 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:00.179236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e0d7-b45f-4c22-82a7-e0f1c324b189 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:00.179236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e0d7-b45f-4c22-82a7-e0f1c324b189 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:00.180607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 657/1329] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:00.185659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969b70b-8ce4-420d-b027-8cb246cddadc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:00.186117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969b70b-8ce4-420d-b027-8cb246cddadc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:00.186222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969b70b-8ce4-420d-b027-8cb246cddadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:00.250347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8969b70b-8ce4-420d-b027-8cb246cddadc None None] MFA Rules not processed for user `47aae071f79647d0a619a13ffb7ae739`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:00.251549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8969b70b-8ce4-420d-b027-8cb246cddadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:00.333764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:00.367410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 673/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:00.652159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e0d7-b45f-4c22-82a7-e0f1c324b189 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:00.653707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4066e0d7-b45f-4c22-82a7-e0f1c324b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:00.758447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e0d7-b45f-4c22-82a7-e0f1c324b189 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:00.777822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:00.831641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e799e-fdef-4d26-aa08-cca177b9a861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:00.832281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e799e-fdef-4d26-aa08-cca177b9a861 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:00.832560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e799e-fdef-4d26-aa08-cca177b9a861 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:00.832829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e799e-fdef-4d26-aa08-cca177b9a861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:00.834445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6f2e799e-fdef-4d26-aa08-cca177b9a861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:00.955100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e799e-fdef-4d26-aa08-cca177b9a861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:00.956574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e799e-fdef-4d26-aa08-cca177b9a861 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:00.977314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 658/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:03.280521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caae40-23d8-4172-a47a-abfe7053ff82 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:03.280911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caae40-23d8-4172-a47a-abfe7053ff82 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:03.281065 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caae40-23d8-4172-a47a-abfe7053ff82 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:03.282416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 674/1332] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:03.288531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb23c4-c38f-42e2-886c-7a28144275af None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:03.288965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb23c4-c38f-42e2-886c-7a28144275af None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:03.289164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb23c4-c38f-42e2-886c-7a28144275af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:03.352515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:03.355838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cabb23c4-c38f-42e2-886c-7a28144275af None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:03.357821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cabb23c4-c38f-42e2-886c-7a28144275af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:03.455066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 659/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:03.734508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caae40-23d8-4172-a47a-abfe7053ff82 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:03.736079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-15caae40-23d8-4172-a47a-abfe7053ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:03.837547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caae40-23d8-4172-a47a-abfe7053ff82 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:03.908810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaae3ba-7326-4b0e-9385-315e67790f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:03.909403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae3ba-7326-4b0e-9385-315e67790f91 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:03.909681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae3ba-7326-4b0e-9385-315e67790f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:03.909956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae3ba-7326-4b0e-9385-315e67790f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:03.911596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-acaae3ba-7326-4b0e-9385-315e67790f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:03.937579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:04.018478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaae3ba-7326-4b0e-9385-315e67790f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:04.020519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaae3ba-7326-4b0e-9385-315e67790f91 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:04.038018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 675/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:05.971572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea115bf-917e-41b0-8059-da3c64239729 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:05.972146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea115bf-917e-41b0-8059-da3c64239729 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:05.972146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea115bf-917e-41b0-8059-da3c64239729 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:05.973517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 660/1335] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:05.978618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95f081-fb13-4f00-83a6-6223b8389091 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:05.979035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95f081-fb13-4f00-83a6-6223b8389091 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:05.979163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95f081-fb13-4f00-83a6-6223b8389091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:06.044405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6b95f081-fb13-4f00-83a6-6223b8389091 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:06.045595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6b95f081-fb13-4f00-83a6-6223b8389091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:06.117023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:06.138748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 676/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:06.425043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea115bf-917e-41b0-8059-da3c64239729 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:06.426580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-aea115bf-917e-41b0-8059-da3c64239729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:06.525892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea115bf-917e-41b0-8059-da3c64239729 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:06.554713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:06.600334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59b282-4ef1-4bf2-bc7f-65e0bdcd18e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:06.600944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b282-4ef1-4bf2-bc7f-65e0bdcd18e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:06.601225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b282-4ef1-4bf2-bc7f-65e0bdcd18e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:06.601497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b282-4ef1-4bf2-bc7f-65e0bdcd18e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:06.603085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6f59b282-4ef1-4bf2-bc7f-65e0bdcd18e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:06.709073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59b282-4ef1-4bf2-bc7f-65e0bdcd18e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:06.710513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59b282-4ef1-4bf2-bc7f-65e0bdcd18e5 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:06.728633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 661/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:08.924010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd0cc2-2164-4ec8-9506-d58110707841 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:08.924567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd0cc2-2164-4ec8-9506-d58110707841 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:08.924567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd0cc2-2164-4ec8-9506-d58110707841 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:08.926422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 677/1338] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:08.931400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930ec7-f348-411a-9da3-0da0803ba718 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:08.931825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930ec7-f348-411a-9da3-0da0803ba718 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:08.931980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930ec7-f348-411a-9da3-0da0803ba718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:08.997326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ff930ec7-f348-411a-9da3-0da0803ba718 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:08.998525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ff930ec7-f348-411a-9da3-0da0803ba718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:09.007450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:09.104550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 662/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:09.382844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd0cc2-2164-4ec8-9506-d58110707841 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:09.384416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b9dd0cc2-2164-4ec8-9506-d58110707841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:09.485566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd0cc2-2164-4ec8-9506-d58110707841 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:09.555819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee50f9-e40a-4467-a289-24cf39718cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:09.556412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee50f9-e40a-4467-a289-24cf39718cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:09.556691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee50f9-e40a-4467-a289-24cf39718cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:09.556959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee50f9-e40a-4467-a289-24cf39718cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:09.558541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-21ee50f9-e40a-4467-a289-24cf39718cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:09.594193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:09.665521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee50f9-e40a-4467-a289-24cf39718cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:09.666942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee50f9-e40a-4467-a289-24cf39718cf0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:09.684223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 678/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:13.554678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f28df1-c4d2-4688-9ca9-c0347aeef30c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:13.555166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f28df1-c4d2-4688-9ca9-c0347aeef30c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:13.555280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f28df1-c4d2-4688-9ca9-c0347aeef30c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:13.556639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 663/1341] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:13.561813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce461f-c3e7-4cce-acfd-e1fbb34aca13 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:13.562206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce461f-c3e7-4cce-acfd-e1fbb34aca13 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:13.562366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce461f-c3e7-4cce-acfd-e1fbb34aca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:13.632402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-87ce461f-c3e7-4cce-acfd-e1fbb34aca13 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:13.633632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-87ce461f-c3e7-4cce-acfd-e1fbb34aca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:13.719603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:13.731873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 679/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:14.027922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f28df1-c4d2-4688-9ca9-c0347aeef30c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:14.029517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f9f28df1-c4d2-4688-9ca9-c0347aeef30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:14.134620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f28df1-c4d2-4688-9ca9-c0347aeef30c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:14.172670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:14.208606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b378a-c2a4-4084-83cb-e30dd2cda191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:14.209208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b378a-c2a4-4084-83cb-e30dd2cda191 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:14.209485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b378a-c2a4-4084-83cb-e30dd2cda191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:14.209750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b378a-c2a4-4084-83cb-e30dd2cda191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:14.211323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-772b378a-c2a4-4084-83cb-e30dd2cda191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:14.321318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b378a-c2a4-4084-83cb-e30dd2cda191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:14.322669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b378a-c2a4-4084-83cb-e30dd2cda191 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:14.341038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 664/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:17.060001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3229db-4223-4109-bac3-71f28733d61d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:17.060527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3229db-4223-4109-bac3-71f28733d61d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:17.060527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3229db-4223-4109-bac3-71f28733d61d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:17.061908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 680/1344] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:17.067125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb14d2-92a2-4e1e-af4c-1468bee554cd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:17.067509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb14d2-92a2-4e1e-af4c-1468bee554cd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:17.067718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb14d2-92a2-4e1e-af4c-1468bee554cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:17.133716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6cbb14d2-92a2-4e1e-af4c-1468bee554cd None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:17.134816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6cbb14d2-92a2-4e1e-af4c-1468bee554cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:17.153204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:17.236399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 665/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:17.516746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3229db-4223-4109-bac3-71f28733d61d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:17.518302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e3229db-4223-4109-bac3-71f28733d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:17.623847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3229db-4223-4109-bac3-71f28733d61d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:17.696465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce2651-b906-4382-a8ff-41a8455d8064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:17.697069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2651-b906-4382-a8ff-41a8455d8064 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:17.697343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2651-b906-4382-a8ff-41a8455d8064 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:17.697611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2651-b906-4382-a8ff-41a8455d8064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:17.699181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-61ce2651-b906-4382-a8ff-41a8455d8064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:17.745691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:17.813512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce2651-b906-4382-a8ff-41a8455d8064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:17.814955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce2651-b906-4382-a8ff-41a8455d8064 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:17.835740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 681/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:21.961966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1315f-7af1-48e1-a919-c0cb85eaeacc None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:21.962505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1315f-7af1-48e1-a919-c0cb85eaeacc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:21.962505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1315f-7af1-48e1-a919-c0cb85eaeacc None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:21.963865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 666/1347] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:21.969244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d855b6a-401d-486b-b805-576efa549013 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:21.969658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d855b6a-401d-486b-b805-576efa549013 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:21.969774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d855b6a-401d-486b-b805-576efa549013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:22.040787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6d855b6a-401d-486b-b805-576efa549013 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:22.042043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d855b6a-401d-486b-b805-576efa549013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:22.140118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:22.143137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 682/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:22.425769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1315f-7af1-48e1-a919-c0cb85eaeacc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:22.427304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32f1315f-7af1-48e1-a919-c0cb85eaeacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:22.527200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1315f-7af1-48e1-a919-c0cb85eaeacc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:22.575112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:22.600707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18404c-6f30-4f9a-b7d3-e3b846f5a209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:22.601305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18404c-6f30-4f9a-b7d3-e3b846f5a209 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:22.601581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18404c-6f30-4f9a-b7d3-e3b846f5a209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:22.601845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18404c-6f30-4f9a-b7d3-e3b846f5a209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:22.603417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9c18404c-6f30-4f9a-b7d3-e3b846f5a209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:22.709615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18404c-6f30-4f9a-b7d3-e3b846f5a209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:22.711049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18404c-6f30-4f9a-b7d3-e3b846f5a209 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:22.728963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 667/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:27.716679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84760555-1f7c-4a90-8be7-6a40ce51111e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:27.716679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84760555-1f7c-4a90-8be7-6a40ce51111e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:27.716679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84760555-1f7c-4a90-8be7-6a40ce51111e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:27.716679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 683/1350] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:27.727471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff685f-5a8d-480a-bd06-dffe666ae2df None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:27.727471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff685f-5a8d-480a-bd06-dffe666ae2df None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:27.727471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff685f-5a8d-480a-bd06-dffe666ae2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:27.849737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-06ff685f-5a8d-480a-bd06-dffe666ae2df None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:27.850904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06ff685f-5a8d-480a-bd06-dffe666ae2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:27.894819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:27.975883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 668/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:04:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:32.024345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa4f54-7390-4bef-951a-02a8902df075 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:32.024906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa4f54-7390-4bef-951a-02a8902df075 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:32.024906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa4f54-7390-4bef-951a-02a8902df075 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:32.026399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 684/1352] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:32.031333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b546ab3-64f7-48b4-a4e7-52ca8c02bae4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:32.032022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b546ab3-64f7-48b4-a4e7-52ca8c02bae4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:32.032492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b546ab3-64f7-48b4-a4e7-52ca8c02bae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:32.113504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3b546ab3-64f7-48b4-a4e7-52ca8c02bae4 None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:32.114977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3b546ab3-64f7-48b4-a4e7-52ca8c02bae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:32.189543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:32.229868 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 669/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:04:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:33.663422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6cd7-b9d4-416a-b8ca-b2691a298332 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:33.663990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6cd7-b9d4-416a-b8ca-b2691a298332 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:33.663990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6cd7-b9d4-416a-b8ca-b2691a298332 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:33.665619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 685/1354] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:33.670903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bc2a6-93ae-4324-9338-390c8ac001a5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:33.671325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bc2a6-93ae-4324-9338-390c8ac001a5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:33.671608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bc2a6-93ae-4324-9338-390c8ac001a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:33.749023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4f1bc2a6-93ae-4324-9338-390c8ac001a5 None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:33.750233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4f1bc2a6-93ae-4324-9338-390c8ac001a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:33.858131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 670/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:04:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:37.265434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c07b2-9072-4cc8-8c99-800ff65c344e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:37.265926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c07b2-9072-4cc8-8c99-800ff65c344e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:37.266042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c07b2-9072-4cc8-8c99-800ff65c344e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:37.267316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 686/1356] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:37.273092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd09904-163e-4686-b2a6-8e02b3558dd6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:37.273563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd09904-163e-4686-b2a6-8e02b3558dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:37.273645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd09904-163e-4686-b2a6-8e02b3558dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:37.324917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:37.361190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5bd09904-163e-4686-b2a6-8e02b3558dd6 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:37.362923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5bd09904-163e-4686-b2a6-8e02b3558dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:37.477248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 671/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:37.779890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06159e97-f49d-4261-a4d6-560bcbee0c59 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:37.780964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06159e97-f49d-4261-a4d6-560bcbee0c59 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:37.782035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06159e97-f49d-4261-a4d6-560bcbee0c59 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:37.785437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 687/1358] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 6 01:04:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:37.793794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef660b9e-f6a7-4a1d-9aaa-87e368061c2f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:37.794738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef660b9e-f6a7-4a1d-9aaa-87e368061c2f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:37.795346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef660b9e-f6a7-4a1d-9aaa-87e368061c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:37.878563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ef660b9e-f6a7-4a1d-9aaa-87e368061c2f None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:37.879745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ef660b9e-f6a7-4a1d-9aaa-87e368061c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:37.929548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:38.001697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 672/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 6 01:04:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:04:38.009252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06159e97-f49d-4261-a4d6-560bcbee0c59 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:38.011941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06159e97-f49d-4261-a4d6-560bcbee0c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:38.163892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06159e97-f49d-4261-a4d6-560bcbee0c59 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:38.240136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:38.291981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac647bc-1d8f-4743-b251-68154d0e95ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:38.293460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac647bc-1d8f-4743-b251-68154d0e95ed service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:38.293930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac647bc-1d8f-4743-b251-68154d0e95ed service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:38.294389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac647bc-1d8f-4743-b251-68154d0e95ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:38.296191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cac647bc-1d8f-4743-b251-68154d0e95ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:38.389479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac647bc-1d8f-4743-b251-68154d0e95ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:38.391340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac647bc-1d8f-4743-b251-68154d0e95ed service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:38.412145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 688/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:40.564922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8908-dbac-4e94-80bf-66bc6765dbbf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:40.565467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8908-dbac-4e94-80bf-66bc6765dbbf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:40.565467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8908-dbac-4e94-80bf-66bc6765dbbf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:40.567023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 673/1361] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:40.572084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9cdbe-4f8b-41c8-b5f4-f003d738e7f1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:40.572532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9cdbe-4f8b-41c8-b5f4-f003d738e7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:40.572623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9cdbe-4f8b-41c8-b5f4-f003d738e7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:40.647609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-38a9cdbe-4f8b-41c8-b5f4-f003d738e7f1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:40.648977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38a9cdbe-4f8b-41c8-b5f4-f003d738e7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:40.694919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:40.768849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 689/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:41.064043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8908-dbac-4e94-80bf-66bc6765dbbf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:41.066069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6fd8908-dbac-4e94-80bf-66bc6765dbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:41.225709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8908-dbac-4e94-80bf-66bc6765dbbf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:41.236246 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:41.357858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820b292-c87f-4c48-a4f4-e77fd1594b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:41.358704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820b292-c87f-4c48-a4f4-e77fd1594b40 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:41.359139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820b292-c87f-4c48-a4f4-e77fd1594b40 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:41.359583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820b292-c87f-4c48-a4f4-e77fd1594b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:41.361520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8820b292-c87f-4c48-a4f4-e77fd1594b40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:41.466797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820b292-c87f-4c48-a4f4-e77fd1594b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:41.468568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820b292-c87f-4c48-a4f4-e77fd1594b40 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:41.489001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 674/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:43.813527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a63a2-8340-4639-89fd-14f431f019ee None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:43.814052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a63a2-8340-4639-89fd-14f431f019ee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:43.814052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a63a2-8340-4639-89fd-14f431f019ee None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:43.815356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 690/1364] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:43.820614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83d94-19a2-41d3-85a7-f3e9270b0499 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:43.821005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83d94-19a2-41d3-85a7-f3e9270b0499 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:43.821171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83d94-19a2-41d3-85a7-f3e9270b0499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:43.836970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:43.888961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-32d83d94-19a2-41d3-85a7-f3e9270b0499 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:43.890156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32d83d94-19a2-41d3-85a7-f3e9270b0499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:43.995815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 675/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:44.292267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f36ab-6e0a-4b52-a552-b68dc73b5403 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:44.292712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f36ab-6e0a-4b52-a552-b68dc73b5403 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:44.293064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f36ab-6e0a-4b52-a552-b68dc73b5403 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:44.294811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 691/1366] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 6 01:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:44.300322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb37991-e59c-4042-aa1d-68dd8a51e0ca None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:44.300706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb37991-e59c-4042-aa1d-68dd8a51e0ca None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:44.301057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb37991-e59c-4042-aa1d-68dd8a51e0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:44.382015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cbb37991-e59c-4042-aa1d-68dd8a51e0ca None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:44.383168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cbb37991-e59c-4042-aa1d-68dd8a51e0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:44.423472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:44.513397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 676/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 6 01:04:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:44.523225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f36ab-6e0a-4b52-a552-b68dc73b5403 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:44.525176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-004f36ab-6e0a-4b52-a552-b68dc73b5403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:44.683380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f36ab-6e0a-4b52-a552-b68dc73b5403 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:44.704533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:44.812643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17531501-0e51-4f21-a799-3bfe18e853b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:44.813249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531501-0e51-4f21-a799-3bfe18e853b8 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:44.813531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531501-0e51-4f21-a799-3bfe18e853b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:44.814108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17531501-0e51-4f21-a799-3bfe18e853b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:44.815966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-17531501-0e51-4f21-a799-3bfe18e853b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:44.905656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17531501-0e51-4f21-a799-3bfe18e853b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:44.907329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17531501-0e51-4f21-a799-3bfe18e853b8 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:44.931938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 692/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:04:45.389335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e739-7523-4a03-bf35-fe4b3f31d9fd None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:45.389880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e739-7523-4a03-bf35-fe4b3f31d9fd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:45.389880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311e739-7523-4a03-bf35-fe4b3f31d9fd None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:45.391270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 677/1369] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:45.397201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00e63f-66bd-46fc-baee-c3dfe3441f7b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:45.397493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00e63f-66bd-46fc-baee-c3dfe3441f7b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:45.397772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00e63f-66bd-46fc-baee-c3dfe3441f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:45.430832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:45.468036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8a00e63f-66bd-46fc-baee-c3dfe3441f7b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:45.469263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a00e63f-66bd-46fc-baee-c3dfe3441f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:45.567246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 693/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:46.152228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea540f5-7f4a-45a8-9563-e4c1e5cd7e5b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:46.152228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea540f5-7f4a-45a8-9563-e4c1e5cd7e5b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:46.152228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea540f5-7f4a-45a8-9563-e4c1e5cd7e5b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:46.153466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 678/1371] 10.222.0.23 () {54 vars in 964 bytes} [Tue Sep 6 01:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:46.157826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75191408-9dda-45f9-85de-563f9e0d7f75 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:46.158319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75191408-9dda-45f9-85de-563f9e0d7f75 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:46.158467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75191408-9dda-45f9-85de-563f9e0d7f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:46.235997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-75191408-9dda-45f9-85de-563f9e0d7f75 None None] MFA Rules not processed for user `1c737d4ca9ea488eb0aad2b328de193c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:46.237755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75191408-9dda-45f9-85de-563f9e0d7f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:46.285363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:46.317926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 694/1372] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Sep 6 01:04:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:46.325581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea540f5-7f4a-45a8-9563-e4c1e5cd7e5b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:46.327091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dea540f5-7f4a-45a8-9563-e4c1e5cd7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:46.435490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea540f5-7f4a-45a8-9563-e4c1e5cd7e5b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:46.477847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:46.516184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bb5c9-7442-4ea8-9cf6-26a97a54dcb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:46.517848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb5c9-7442-4ea8-9cf6-26a97a54dcb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:46.518598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb5c9-7442-4ea8-9cf6-26a97a54dcb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:46.519305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bb5c9-7442-4ea8-9cf6-26a97a54dcb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:46.521096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ea3bb5c9-7442-4ea8-9cf6-26a97a54dcb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:46.656199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bb5c9-7442-4ea8-9cf6-26a97a54dcb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:46.658749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bb5c9-7442-4ea8-9cf6-26a97a54dcb2 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:46.685106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 679/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:46.921512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df25473-d7cd-4e13-96f6-4a41a2244f06 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:46.921795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df25473-d7cd-4e13-96f6-4a41a2244f06 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:46.922095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df25473-d7cd-4e13-96f6-4a41a2244f06 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:46.923442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 695/1374] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:46.928634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6228b4-db75-4bc4-9488-cad0cf8b79ee None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:46.928989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6228b4-db75-4bc4-9488-cad0cf8b79ee None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:46.929337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6228b4-db75-4bc4-9488-cad0cf8b79ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:46.995180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9f6228b4-db75-4bc4-9488-cad0cf8b79ee None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:46.996332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9f6228b4-db75-4bc4-9488-cad0cf8b79ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:47.017313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:47.094232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 680/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:47.380909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df25473-d7cd-4e13-96f6-4a41a2244f06 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:47.382556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3df25473-d7cd-4e13-96f6-4a41a2244f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:47.534631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df25473-d7cd-4e13-96f6-4a41a2244f06 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:47.589778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:47.651395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3155d-5a56-435b-9802-4eec02faf4af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:47.652027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3155d-5a56-435b-9802-4eec02faf4af service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:47.652312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3155d-5a56-435b-9802-4eec02faf4af service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:47.652588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3155d-5a56-435b-9802-4eec02faf4af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:47.654217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67d3155d-5a56-435b-9802-4eec02faf4af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:47.745795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3155d-5a56-435b-9802-4eec02faf4af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:47.747443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3155d-5a56-435b-9802-4eec02faf4af service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:47.765995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 696/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:49.690370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b0ae9-e50e-4b3e-814b-f51401114b1c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:49.690884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b0ae9-e50e-4b3e-814b-f51401114b1c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:49.691005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b0ae9-e50e-4b3e-814b-f51401114b1c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:49.692354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 681/1377] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:49.697524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874ac48-078c-4d61-8ff6-420accbbb69a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:49.697903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874ac48-078c-4d61-8ff6-420accbbb69a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:49.698074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874ac48-078c-4d61-8ff6-420accbbb69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:49.782062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2874ac48-078c-4d61-8ff6-420accbbb69a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:49.783735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2874ac48-078c-4d61-8ff6-420accbbb69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:49.790931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:49.881896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 697/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:50.184929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b0ae9-e50e-4b3e-814b-f51401114b1c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:50.186471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-169b0ae9-e50e-4b3e-814b-f51401114b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:50.354557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b0ae9-e50e-4b3e-814b-f51401114b1c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:50.358854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:50.479920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49084e-bfca-4340-a278-8562a280e0d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:50.480811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49084e-bfca-4340-a278-8562a280e0d4 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:50.481322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49084e-bfca-4340-a278-8562a280e0d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:50.481745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49084e-bfca-4340-a278-8562a280e0d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:50.483590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e49084e-bfca-4340-a278-8562a280e0d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:50.587845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49084e-bfca-4340-a278-8562a280e0d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:50.589858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49084e-bfca-4340-a278-8562a280e0d4 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:50.611253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 682/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:52.583341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9ff1-e4c7-432d-b05f-649d23bf58b6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:52.583814 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9ff1-e4c7-432d-b05f-649d23bf58b6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:52.584165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9ff1-e4c7-432d-b05f-649d23bf58b6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:52.588614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 698/1380] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:52.591196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e660e6-bf4f-4e54-a4e9-6d4721fdc0d8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:52.591532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e660e6-bf4f-4e54-a4e9-6d4721fdc0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:52.591532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e660e6-bf4f-4e54-a4e9-6d4721fdc0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:52.603442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:52.661488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-96e660e6-bf4f-4e54-a4e9-6d4721fdc0d8 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:52.662844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-96e660e6-bf4f-4e54-a4e9-6d4721fdc0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:52.767904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 683/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:53.063813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb9ff1-e4c7-432d-b05f-649d23bf58b6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:53.065421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ccb9ff1-e4c7-432d-b05f-649d23bf58b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:53.222225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:53.233433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb9ff1-e4c7-432d-b05f-649d23bf58b6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:53.354105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9966d-27d3-492c-9d02-ecb640cb6f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:53.354734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9966d-27d3-492c-9d02-ecb640cb6f05 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:53.355088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9966d-27d3-492c-9d02-ecb640cb6f05 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:53.355434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9966d-27d3-492c-9d02-ecb640cb6f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:53.357119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a4c9966d-27d3-492c-9d02-ecb640cb6f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:53.451133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9966d-27d3-492c-9d02-ecb640cb6f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:53.453046 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9966d-27d3-492c-9d02-ecb640cb6f05 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:53.464642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:53.474519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 699/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:55.438427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb346b1-8751-43dd-99c6-bf2f917b1c00 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:55.439055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb346b1-8751-43dd-99c6-bf2f917b1c00 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:55.439055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb346b1-8751-43dd-99c6-bf2f917b1c00 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:55.440426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 684/1383] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:55.446201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33cca0-39f9-4e5f-9f54-929c5ff54d11 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:55.446588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33cca0-39f9-4e5f-9f54-929c5ff54d11 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:55.446959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33cca0-39f9-4e5f-9f54-929c5ff54d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:55.521562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4e33cca0-39f9-4e5f-9f54-929c5ff54d11 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:55.522665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e33cca0-39f9-4e5f-9f54-929c5ff54d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:55.639672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 700/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:55.936290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb346b1-8751-43dd-99c6-bf2f917b1c00 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:55.938416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fbb346b1-8751-43dd-99c6-bf2f917b1c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:56.046601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:56.123542 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb346b1-8751-43dd-99c6-bf2f917b1c00 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:56.260206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821cd22-7eb0-4f9f-886b-7d3d24e6a457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:56.260710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821cd22-7eb0-4f9f-886b-7d3d24e6a457 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:56.261017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821cd22-7eb0-4f9f-886b-7d3d24e6a457 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:56.261253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821cd22-7eb0-4f9f-886b-7d3d24e6a457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:56.262835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4821cd22-7eb0-4f9f-886b-7d3d24e6a457 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:56.311978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:56.350773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821cd22-7eb0-4f9f-886b-7d3d24e6a457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:56.352397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821cd22-7eb0-4f9f-886b-7d3d24e6a457 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:56.370674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 685/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:04:58.279699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cbc94-c442-4faa-916d-39d3c04ac299 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:58.280263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cbc94-c442-4faa-916d-39d3c04ac299 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:58.280263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cbc94-c442-4faa-916d-39d3c04ac299 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:58.281620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 701/1386] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:04:58.287028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595b878-18e9-4d4f-a1a2-e16c0c021d80 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:58.287456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595b878-18e9-4d4f-a1a2-e16c0c021d80 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:58.287550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595b878-18e9-4d4f-a1a2-e16c0c021d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:58.364356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0595b878-18e9-4d4f-a1a2-e16c0c021d80 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:04:58.365574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0595b878-18e9-4d4f-a1a2-e16c0c021d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:58.462636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 686/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:04:58.753091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cbc94-c442-4faa-916d-39d3c04ac299 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:04:58.754642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-513cbc94-c442-4faa-916d-39d3c04ac299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:58.832568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:58.905331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cbc94-c442-4faa-916d-39d3c04ac299 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:04:59.019320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47442f7-c879-495d-8579-8456ca2e79fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:04:59.019885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47442f7-c879-495d-8579-8456ca2e79fe service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:04:59.020162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47442f7-c879-495d-8579-8456ca2e79fe service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:04:59.020493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47442f7-c879-495d-8579-8456ca2e79fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:04:59.022032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d47442f7-c879-495d-8579-8456ca2e79fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:04:59.062135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:04:59.113870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47442f7-c879-495d-8579-8456ca2e79fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:04:59.115923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47442f7-c879-495d-8579-8456ca2e79fe service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:04:59.135037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 702/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:01.077878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044fe86-ca49-44c6-a982-070ee8fa2753 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:01.078438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044fe86-ca49-44c6-a982-070ee8fa2753 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:01.078438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044fe86-ca49-44c6-a982-070ee8fa2753 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:01.080521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 687/1389] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:01.085537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7a46a-f812-4b08-be18-ba35431bbf16 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:01.086001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7a46a-f812-4b08-be18-ba35431bbf16 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:01.086285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7a46a-f812-4b08-be18-ba35431bbf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:01.155544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5fd7a46a-f812-4b08-be18-ba35431bbf16 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:01.156793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5fd7a46a-f812-4b08-be18-ba35431bbf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:01.264463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 703/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:01.563370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044fe86-ca49-44c6-a982-070ee8fa2753 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:01.565045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e044fe86-ca49-44c6-a982-070ee8fa2753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:01.602185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:01.681127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9429-9ce9-4403-acf6-3b4c820f9aa5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:01.681420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9429-9ce9-4403-acf6-3b4c820f9aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:01.681694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9429-9ce9-4403-acf6-3b4c820f9aa5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:01.683032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 704/1391] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 01:05:01.687998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47c431-9dd9-4479-9e46-41f312bfded5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:01.688289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47c431-9dd9-4479-9e46-41f312bfded5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:01.688643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47c431-9dd9-4479-9e46-41f312bfded5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:01.703656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:01.718676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044fe86-ca49-44c6-a982-070ee8fa2753 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:01.757543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ac47c431-9dd9-4479-9e46-41f312bfded5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:01.758791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ac47c431-9dd9-4479-9e46-41f312bfded5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:01.821994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:01.834471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03fb4b-4019-4bdf-887e-cccb5d4a1716 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:01.835076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03fb4b-4019-4bdf-887e-cccb5d4a1716 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:01.835354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03fb4b-4019-4bdf-887e-cccb5d4a1716 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:01.835624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03fb4b-4019-4bdf-887e-cccb5d4a1716 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:01.837269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4d03fb4b-4019-4bdf-887e-cccb5d4a1716 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:01.855531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 705/1392] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:01.935472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03fb4b-4019-4bdf-887e-cccb5d4a1716 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:01.937101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03fb4b-4019-4bdf-887e-cccb5d4a1716 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:01.955894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 688/1393] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:02.160343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431118c1-7f7b-4bae-876b-2ea59dcb0aa5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:02.160898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431118c1-7f7b-4bae-876b-2ea59dcb0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:02.160898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431118c1-7f7b-4bae-876b-2ea59dcb0aa5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:02.162222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 706/1394] 10.222.0.23 () {54 vars in 961 bytes} [Tue Sep 6 01:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:02.167591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295530e-21e8-4d9c-8017-abb0fe873d74 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:02.168066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295530e-21e8-4d9c-8017-abb0fe873d74 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:02.168169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295530e-21e8-4d9c-8017-abb0fe873d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:02.246259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9295530e-21e8-4d9c-8017-abb0fe873d74 None None] MFA Rules not processed for user `10bbe5bd5324482b9a74836a552a077d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:02.247469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9295530e-21e8-4d9c-8017-abb0fe873d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:02.302097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:02.327275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 689/1395] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Sep 6 01:05:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:02.336504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431118c1-7f7b-4bae-876b-2ea59dcb0aa5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:02.338169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-431118c1-7f7b-4bae-876b-2ea59dcb0aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:02.442195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:02.449873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431118c1-7f7b-4bae-876b-2ea59dcb0aa5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:02.525194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079413f-d808-41f9-89e2-4e6d5515d610 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:02.525779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079413f-d808-41f9-89e2-4e6d5515d610 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:02.526058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079413f-d808-41f9-89e2-4e6d5515d610 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:02.526330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079413f-d808-41f9-89e2-4e6d5515d610 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:02.528029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7079413f-d808-41f9-89e2-4e6d5515d610 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:02.638684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079413f-d808-41f9-89e2-4e6d5515d610 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:02.640150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079413f-d808-41f9-89e2-4e6d5515d610 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:02.658888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 707/1396] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:03.393036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d331-ae1a-4d3f-b394-deb2678a9d35 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:03.393511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d331-ae1a-4d3f-b394-deb2678a9d35 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:03.393633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64d331-ae1a-4d3f-b394-deb2678a9d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:03.460321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ed64d331-ae1a-4d3f-b394-deb2678a9d35 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:03.461517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ed64d331-ae1a-4d3f-b394-deb2678a9d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:03.552030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 690/1397] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:05:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:03.600245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3514a-adfe-4221-84f6-a2a07ee71267 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:03.600733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3514a-adfe-4221-84f6-a2a07ee71267 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:03.600847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3514a-adfe-4221-84f6-a2a07ee71267 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:03.602190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 708/1398] 10.222.0.23 () {54 vars in 943 bytes} [Tue Sep 6 01:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:03.608471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868fc3f-e6e8-4a00-a96e-f4a05b79ea34 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:03.608760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868fc3f-e6e8-4a00-a96e-f4a05b79ea34 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:03.609058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868fc3f-e6e8-4a00-a96e-f4a05b79ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:03.687955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9868fc3f-e6e8-4a00-a96e-f4a05b79ea34 None None] MFA Rules not processed for user `e26430820d684ca8b4f63e473a16b6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:03.689796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9868fc3f-e6e8-4a00-a96e-f4a05b79ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:03.702678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:03.796743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 691/1399] 10.222.0.23 () {58 vars in 1040 bytes} [Tue Sep 6 01:05:03 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:05:03.806820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3514a-adfe-4221-84f6-a2a07ee71267 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:03.808406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f0c3514a-adfe-4221-84f6-a2a07ee71267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:03.832645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:03.893759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5272d-34fe-4108-a795-f4282473126c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:03.894047 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5272d-34fe-4108-a795-f4282473126c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:03.894328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5272d-34fe-4108-a795-f4282473126c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:03.895719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 692/1400] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 01:05:03.900863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:03.901149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:03.901421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:03.949133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3514a-adfe-4221-84f6-a2a07ee71267 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:03.967562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:03.968799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:04.044300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:04.053678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ab3f0-53dd-41a3-9538-4bb9898772d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:04.054222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ab3f0-53dd-41a3-9538-4bb9898772d7 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:04.054499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ab3f0-53dd-41a3-9538-4bb9898772d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:04.054765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ab3f0-53dd-41a3-9538-4bb9898772d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:04.056423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f0ab3f0-53dd-41a3-9538-4bb9898772d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:04.063715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:04.069880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 693/1401] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:05:04.140995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ab3f0-53dd-41a3-9538-4bb9898772d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:04.143089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ab3f0-53dd-41a3-9538-4bb9898772d7 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:04.150918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 709/1402] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:05:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:05:04.366307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:04.367641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:04.488492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214b24f-abb3-4def-aea0-a0bad88c2822 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:04.603405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:04.633818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edc6cf-941d-48c1-a741-f533fb2625f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:04.634351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edc6cf-941d-48c1-a741-f533fb2625f0 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:04.634630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edc6cf-941d-48c1-a741-f533fb2625f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:04.634906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edc6cf-941d-48c1-a741-f533fb2625f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:04.636595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e1edc6cf-941d-48c1-a741-f533fb2625f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:04.739695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edc6cf-941d-48c1-a741-f533fb2625f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:04.741341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edc6cf-941d-48c1-a741-f533fb2625f0 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:04.760471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 694/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:06.775364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ad27b-bbbb-4450-b80a-1564461c66ba None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:06.775364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ad27b-bbbb-4450-b80a-1564461c66ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:06.775364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ad27b-bbbb-4450-b80a-1564461c66ba None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:06.775364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 710/1404] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:06.780627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5708df1-7720-4346-9f40-75f16b730c83 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:06.781125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5708df1-7720-4346-9f40-75f16b730c83 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:06.781233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5708df1-7720-4346-9f40-75f16b730c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:06.853106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f5708df1-7720-4346-9f40-75f16b730c83 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:06.854326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f5708df1-7720-4346-9f40-75f16b730c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:06.909396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:06.961670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 695/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:07.262953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ad27b-bbbb-4450-b80a-1564461c66ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:07.264532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e68ad27b-bbbb-4450-b80a-1564461c66ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:07.435518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ad27b-bbbb-4450-b80a-1564461c66ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:07.438988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:07.567742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa411b-7825-4fe0-b1f2-8de54ba69b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:07.568648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa411b-7825-4fe0-b1f2-8de54ba69b7d service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:07.568997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa411b-7825-4fe0-b1f2-8de54ba69b7d service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:07.569282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa411b-7825-4fe0-b1f2-8de54ba69b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:07.571171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4bfa411b-7825-4fe0-b1f2-8de54ba69b7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:07.676359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa411b-7825-4fe0-b1f2-8de54ba69b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:07.677977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa411b-7825-4fe0-b1f2-8de54ba69b7d service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:07.699457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 711/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:09.681227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7a9d-17f6-400e-8c4e-4ecdceb6cc86 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:09.681775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7a9d-17f6-400e-8c4e-4ecdceb6cc86 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:09.681775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a7a9d-17f6-400e-8c4e-4ecdceb6cc86 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:09.683786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 696/1407] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:09.688939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc805125-6384-4266-992d-1bb397a2ed09 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:09.689385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc805125-6384-4266-992d-1bb397a2ed09 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:09.689490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc805125-6384-4266-992d-1bb397a2ed09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:09.699084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:09.762786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dc805125-6384-4266-992d-1bb397a2ed09 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:09.764148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dc805125-6384-4266-992d-1bb397a2ed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:09.872127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 712/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:10.164700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a7a9d-17f6-400e-8c4e-4ecdceb6cc86 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:10.166378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-955a7a9d-17f6-400e-8c4e-4ecdceb6cc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:10.328514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a7a9d-17f6-400e-8c4e-4ecdceb6cc86 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:10.368124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:10.453451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01524c-7950-4e12-a6c8-54996645262c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:10.454652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01524c-7950-4e12-a6c8-54996645262c service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:10.455057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01524c-7950-4e12-a6c8-54996645262c service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:10.455335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01524c-7950-4e12-a6c8-54996645262c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:10.457106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cd01524c-7950-4e12-a6c8-54996645262c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:10.562111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01524c-7950-4e12-a6c8-54996645262c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:10.565236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01524c-7950-4e12-a6c8-54996645262c service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:10.617072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 697/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:19.870283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=62 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:20.613582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=82 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:26.199956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d381-a2a5-4fdf-bc7f-cde8ef8d0248 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:26.200499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d381-a2a5-4fdf-bc7f-cde8ef8d0248 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:26.200499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d381-a2a5-4fdf-bc7f-cde8ef8d0248 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:26.201852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 713/1410] 10.222.0.23 () {54 vars in 918 bytes} [Tue Sep 6 01:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:26.207448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545556d-1f35-4a22-8d54-5d630ab9e47b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:26.207947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545556d-1f35-4a22-8d54-5d630ab9e47b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:26.208050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545556d-1f35-4a22-8d54-5d630ab9e47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:26.281649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c545556d-1f35-4a22-8d54-5d630ab9e47b None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:26.282835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c545556d-1f35-4a22-8d54-5d630ab9e47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:26.386185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 698/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 6 01:05:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:05:26.400653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508f365-1b39-4625-91d2-b8a4c2d836b1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:26.400969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508f365-1b39-4625-91d2-b8a4c2d836b1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:26.401252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508f365-1b39-4625-91d2-b8a4c2d836b1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:26.402574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 714/1412] 10.222.0.23 () {54 vars in 918 bytes} [Tue Sep 6 01:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:26.407015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4aa26-6811-46f9-b288-201f7931ee29 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:26.407301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4aa26-6811-46f9-b288-201f7931ee29 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:26.407580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4aa26-6811-46f9-b288-201f7931ee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:26.484610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:26.487389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f0e4aa26-6811-46f9-b288-201f7931ee29 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:26.489354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f0e4aa26-6811-46f9-b288-201f7931ee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:26.606181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 699/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 6 01:05:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:28.971511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f522c0-7e5f-4557-8637-cb7b93c87a66 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:28.972111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f522c0-7e5f-4557-8637-cb7b93c87a66 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:28.972111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f522c0-7e5f-4557-8637-cb7b93c87a66 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:28.973443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:28.978638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057efdac-355c-4a13-a145-224175b4b947 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:28.979039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057efdac-355c-4a13-a145-224175b4b947 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:28.979180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057efdac-355c-4a13-a145-224175b4b947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:29.046059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-057efdac-355c-4a13-a145-224175b4b947 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:29.047291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-057efdac-355c-4a13-a145-224175b4b947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:29.100779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:29.141859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:29.466274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f522c0-7e5f-4557-8637-cb7b93c87a66 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:29.469829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04f522c0-7e5f-4557-8637-cb7b93c87a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:29.627231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f522c0-7e5f-4557-8637-cb7b93c87a66 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:29.699350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:29.739391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfbef1-150c-49b4-b6bb-3a6c0dfb0b0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:29.739960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbef1-150c-49b4-b6bb-3a6c0dfb0b0c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:29.740240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbef1-150c-49b4-b6bb-3a6c0dfb0b0c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:29.740517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbef1-150c-49b4-b6bb-3a6c0dfb0b0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:29.742116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9dbfbef1-150c-49b4-b6bb-3a6c0dfb0b0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:29.829339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfbef1-150c-49b4-b6bb-3a6c0dfb0b0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:29.830915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfbef1-150c-49b4-b6bb-3a6c0dfb0b0c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:29.848372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 716/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:30.736134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac30f7-cc7a-47f0-86e3-4c6182bfd289 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:30.736701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac30f7-cc7a-47f0-86e3-4c6182bfd289 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:30.736701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac30f7-cc7a-47f0-86e3-4c6182bfd289 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:30.738026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:30.742868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc72f-1f23-4a28-8a15-1945661f7bad None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:30.743255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc72f-1f23-4a28-8a15-1945661f7bad None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:30.743414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc72f-1f23-4a28-8a15-1945661f7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:30.817940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-869bc72f-1f23-4a28-8a15-1945661f7bad None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:30.819127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-869bc72f-1f23-4a28-8a15-1945661f7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:30.845295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:30.920058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:31.069272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac30f7-cc7a-47f0-86e3-4c6182bfd289 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:31.070778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8aac30f7-cc7a-47f0-86e3-4c6182bfd289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:31.216417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac30f7-cc7a-47f0-86e3-4c6182bfd289 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:31.256400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:31.331710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690deccf-5e98-45b9-876a-a3716f003c61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:31.332345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690deccf-5e98-45b9-876a-a3716f003c61 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:31.332627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690deccf-5e98-45b9-876a-a3716f003c61 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:31.332902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690deccf-5e98-45b9-876a-a3716f003c61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:31.334514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-690deccf-5e98-45b9-876a-a3716f003c61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:31.421087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690deccf-5e98-45b9-876a-a3716f003c61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:31.423191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690deccf-5e98-45b9-876a-a3716f003c61 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:31.441193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 702/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:32.314481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6fb2d-5bb7-4b8f-a54e-fa6e9582568a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:32.315061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6fb2d-5bb7-4b8f-a54e-fa6e9582568a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:32.315061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6fb2d-5bb7-4b8f-a54e-fa6e9582568a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:32.316487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:32.321568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d46f8-7095-44bf-aafb-05016ec5022b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:32.321969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d46f8-7095-44bf-aafb-05016ec5022b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:32.322110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d46f8-7095-44bf-aafb-05016ec5022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:32.376474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:32.398311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3f3d46f8-7095-44bf-aafb-05016ec5022b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:32.399526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f3d46f8-7095-44bf-aafb-05016ec5022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:32.494258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:32.649490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6fb2d-5bb7-4b8f-a54e-fa6e9582568a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:32.651045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-11a6fb2d-5bb7-4b8f-a54e-fa6e9582568a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:32.795588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6fb2d-5bb7-4b8f-a54e-fa6e9582568a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:32.808264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:32.909434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a255681-fcb7-4c9e-8c4f-cdfc4fd43b95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:32.909933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a255681-fcb7-4c9e-8c4f-cdfc4fd43b95 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:32.910209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a255681-fcb7-4c9e-8c4f-cdfc4fd43b95 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:32.910475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a255681-fcb7-4c9e-8c4f-cdfc4fd43b95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:32.912086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a255681-fcb7-4c9e-8c4f-cdfc4fd43b95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:32.998535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a255681-fcb7-4c9e-8c4f-cdfc4fd43b95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:33.000760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a255681-fcb7-4c9e-8c4f-cdfc4fd43b95 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:33.018522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 719/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:33.875858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad000b4-4982-4fa8-ad92-5db28d00ac00 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:33.876383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad000b4-4982-4fa8-ad92-5db28d00ac00 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:33.876383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad000b4-4982-4fa8-ad92-5db28d00ac00 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:33.877690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:33.882627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c54c4-fb99-4409-a8d7-31ea307c9686 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:33.883019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c54c4-fb99-4409-a8d7-31ea307c9686 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:33.883175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c54c4-fb99-4409-a8d7-31ea307c9686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:33.902959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:33.958198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c18c54c4-fb99-4409-a8d7-31ea307c9686 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:33.959394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c18c54c4-fb99-4409-a8d7-31ea307c9686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:34.052083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:05:34.205335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad000b4-4982-4fa8-ad92-5db28d00ac00 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:34.206876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8ad000b4-4982-4fa8-ad92-5db28d00ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:34.315036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:34.349411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad000b4-4982-4fa8-ad92-5db28d00ac00 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:34.458044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06387d61-b82c-43d7-ae1c-b6efb3dbf93a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:34.458578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06387d61-b82c-43d7-ae1c-b6efb3dbf93a service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:34.458856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06387d61-b82c-43d7-ae1c-b6efb3dbf93a service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:34.459124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06387d61-b82c-43d7-ae1c-b6efb3dbf93a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:34.460745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06387d61-b82c-43d7-ae1c-b6efb3dbf93a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:34.534166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:34.547989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06387d61-b82c-43d7-ae1c-b6efb3dbf93a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:34.549583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06387d61-b82c-43d7-ae1c-b6efb3dbf93a service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:34.566942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 705/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:35.433170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449d75-da87-42c4-b001-7711114530ac None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:35.433697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449d75-da87-42c4-b001-7711114530ac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:35.433697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449d75-da87-42c4-b001-7711114530ac None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:35.435067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:35.440437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac76e2c-5658-42d8-8072-ca9361d0735b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:35.440878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac76e2c-5658-42d8-8072-ca9361d0735b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:35.440991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac76e2c-5658-42d8-8072-ca9361d0735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:35.515172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-aac76e2c-5658-42d8-8072-ca9361d0735b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:35.517198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-aac76e2c-5658-42d8-8072-ca9361d0735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:35.612537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:35.763741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77449d75-da87-42c4-b001-7711114530ac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:35.765299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-77449d75-da87-42c4-b001-7711114530ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:35.850510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:35.914098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77449d75-da87-42c4-b001-7711114530ac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:36.140496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45e758-3be3-4efa-a67a-e368cc0f8c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:36.141029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e758-3be3-4efa-a67a-e368cc0f8c84 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:36.141323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e758-3be3-4efa-a67a-e368cc0f8c84 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:36.141575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e758-3be3-4efa-a67a-e368cc0f8c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:36.143159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b45e758-3be3-4efa-a67a-e368cc0f8c84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:36.192007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:36.238303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45e758-3be3-4efa-a67a-e368cc0f8c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:36.239928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45e758-3be3-4efa-a67a-e368cc0f8c84 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:36.257432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 722/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:37.061623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00a8b-8164-48c1-bba9-ab08d6b27b68 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:37.062161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00a8b-8164-48c1-bba9-ab08d6b27b68 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:37.062161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00a8b-8164-48c1-bba9-ab08d6b27b68 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:37.064043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:37.068812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee133af-d461-49ef-baf4-1fd3c2fce152 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:37.069240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee133af-d461-49ef-baf4-1fd3c2fce152 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:37.069347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee133af-d461-49ef-baf4-1fd3c2fce152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:37.142977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2ee133af-d461-49ef-baf4-1fd3c2fce152 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:37.144200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ee133af-d461-49ef-baf4-1fd3c2fce152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:37.237620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:37.389154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00a8b-8164-48c1-bba9-ab08d6b27b68 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:37.390618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a2b00a8b-8164-48c1-bba9-ab08d6b27b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:37.442502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:37.537876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00a8b-8164-48c1-bba9-ab08d6b27b68 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:37.647986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c591e-0f91-40da-a156-936fbad42a06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:37.648504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c591e-0f91-40da-a156-936fbad42a06 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:37.648783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c591e-0f91-40da-a156-936fbad42a06 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:37.649053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c591e-0f91-40da-a156-936fbad42a06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:37.650642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b80c591e-0f91-40da-a156-936fbad42a06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:37.657951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:37.743563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c591e-0f91-40da-a156-936fbad42a06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:37.745200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c591e-0f91-40da-a156-936fbad42a06 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:37.762593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 708/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:38.584251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbf1c0-ddd9-474b-ae59-d55303c4da90 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:38.584251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbf1c0-ddd9-474b-ae59-d55303c4da90 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:38.584732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbf1c0-ddd9-474b-ae59-d55303c4da90 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:38.585813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:38.590791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7e75b-5530-4606-bcd2-a8ed2c2cc7b8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:38.591224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7e75b-5530-4606-bcd2-a8ed2c2cc7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:38.591335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7e75b-5530-4606-bcd2-a8ed2c2cc7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:38.666678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cca7e75b-5530-4606-bcd2-a8ed2c2cc7b8 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:38.667897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cca7e75b-5530-4606-bcd2-a8ed2c2cc7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:38.733418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:38.763463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:05:38.916949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbf1c0-ddd9-474b-ae59-d55303c4da90 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:38.918420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bdbbf1c0-ddd9-474b-ae59-d55303c4da90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:38.936211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:39.062249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbf1c0-ddd9-474b-ae59-d55303c4da90 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:39.146823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:39.172158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb36d4c-e836-412d-a767-f08dc68b77a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:39.172770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb36d4c-e836-412d-a767-f08dc68b77a6 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:39.173038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb36d4c-e836-412d-a767-f08dc68b77a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:39.173308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb36d4c-e836-412d-a767-f08dc68b77a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:39.174888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cdb36d4c-e836-412d-a767-f08dc68b77a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:39.264354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb36d4c-e836-412d-a767-f08dc68b77a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:39.266548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb36d4c-e836-412d-a767-f08dc68b77a6 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:39.284493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 725/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:40.079563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23663ce0-b7cc-4574-9715-f30570beeb46 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:40.080157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23663ce0-b7cc-4574-9715-f30570beeb46 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:40.080157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23663ce0-b7cc-4574-9715-f30570beeb46 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:40.081525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:40.086574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488ffc8-5ce0-4f75-910e-71019b40be25 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:40.086965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488ffc8-5ce0-4f75-910e-71019b40be25 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:40.087120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488ffc8-5ce0-4f75-910e-71019b40be25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:40.163124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a488ffc8-5ce0-4f75-910e-71019b40be25 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:40.164351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a488ffc8-5ce0-4f75-910e-71019b40be25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:40.204447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:40.257967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:40.407314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23663ce0-b7cc-4574-9715-f30570beeb46 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:40.408794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-23663ce0-b7cc-4574-9715-f30570beeb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:40.552309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23663ce0-b7cc-4574-9715-f30570beeb46 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:40.606206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:40.665002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b7bf3-646a-4b4a-961d-04fe0eece7db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:40.665539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b7bf3-646a-4b4a-961d-04fe0eece7db service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:40.665812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b7bf3-646a-4b4a-961d-04fe0eece7db service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:40.666079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b7bf3-646a-4b4a-961d-04fe0eece7db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:40.667673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dc0b7bf3-646a-4b4a-961d-04fe0eece7db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:40.755926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b7bf3-646a-4b4a-961d-04fe0eece7db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:40.757539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b7bf3-646a-4b4a-961d-04fe0eece7db service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:40.775865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 711/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:41.577243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73653f9-1bc5-4b85-9d25-759c4e3b0ec7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:41.578005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73653f9-1bc5-4b85-9d25-759c4e3b0ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:41.578005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73653f9-1bc5-4b85-9d25-759c4e3b0ec7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:41.579086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:41.584559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca53fa-88b7-457f-a460-3ec8db7b325a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:41.585042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca53fa-88b7-457f-a460-3ec8db7b325a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:41.585137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca53fa-88b7-457f-a460-3ec8db7b325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:41.660725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b5ca53fa-88b7-457f-a460-3ec8db7b325a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:41.661935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b5ca53fa-88b7-457f-a460-3ec8db7b325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:41.663966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:41.755521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:41.909052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73653f9-1bc5-4b85-9d25-759c4e3b0ec7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:41.910478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a73653f9-1bc5-4b85-9d25-759c4e3b0ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:42.056214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73653f9-1bc5-4b85-9d25-759c4e3b0ec7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:42.083236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:42.166680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080e08-750c-480e-b1a7-a482b73c08c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:42.167207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080e08-750c-480e-b1a7-a482b73c08c0 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:42.167486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080e08-750c-480e-b1a7-a482b73c08c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:42.167777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080e08-750c-480e-b1a7-a482b73c08c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:42.169372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e5080e08-750c-480e-b1a7-a482b73c08c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:42.256532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080e08-750c-480e-b1a7-a482b73c08c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:42.258110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080e08-750c-480e-b1a7-a482b73c08c0 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:42.276161 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 728/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:44.091961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733483e-069d-49e1-9e63-6ce2f51f0246 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:44.092521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733483e-069d-49e1-9e63-6ce2f51f0246 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:44.092521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733483e-069d-49e1-9e63-6ce2f51f0246 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:44.093852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:44.099276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b8d5-0f17-4712-b196-a842efb34ba7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:44.099697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b8d5-0f17-4712-b196-a842efb34ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:44.099866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b8d5-0f17-4712-b196-a842efb34ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:44.140108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:44.165343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5047b8d5-0f17-4712-b196-a842efb34ba7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:44.166545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5047b8d5-0f17-4712-b196-a842efb34ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:44.263882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:44.546145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733483e-069d-49e1-9e63-6ce2f51f0246 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:44.548282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d733483e-069d-49e1-9e63-6ce2f51f0246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:44.665004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733483e-069d-49e1-9e63-6ce2f51f0246 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:44.692741 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:44.734049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f8e87-b66a-4448-ba1b-030e53404286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:44.734695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8e87-b66a-4448-ba1b-030e53404286 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:44.734961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8e87-b66a-4448-ba1b-030e53404286 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:44.735234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f8e87-b66a-4448-ba1b-030e53404286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:44.736880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c90f8e87-b66a-4448-ba1b-030e53404286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:44.842590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f8e87-b66a-4448-ba1b-030e53404286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:44.844602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f8e87-b66a-4448-ba1b-030e53404286 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:44.864145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 714/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:46.991588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55841cc-f728-4bae-841d-394e6835ff40 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:46.992173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55841cc-f728-4bae-841d-394e6835ff40 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:46.992275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55841cc-f728-4bae-841d-394e6835ff40 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:46.993579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:46.998685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d427d-ce6a-481a-95f4-842e057e0a88 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:46.999086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d427d-ce6a-481a-95f4-842e057e0a88 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:46.999242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d427d-ce6a-481a-95f4-842e057e0a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:47.064679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8c9d427d-ce6a-481a-95f4-842e057e0a88 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:47.065990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8c9d427d-ce6a-481a-95f4-842e057e0a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:47.078463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:47.160489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:47.444159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55841cc-f728-4bae-841d-394e6835ff40 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:47.445689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e55841cc-f728-4bae-841d-394e6835ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:47.547357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55841cc-f728-4bae-841d-394e6835ff40 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:47.561963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:47.627005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cfb54-fadb-45b1-bda9-74bebe8eaf1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:47.627606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cfb54-fadb-45b1-bda9-74bebe8eaf1b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:47.627910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cfb54-fadb-45b1-bda9-74bebe8eaf1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:47.628178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cfb54-fadb-45b1-bda9-74bebe8eaf1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:47.629742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d74cfb54-fadb-45b1-bda9-74bebe8eaf1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:47.738247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cfb54-fadb-45b1-bda9-74bebe8eaf1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:47.739672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cfb54-fadb-45b1-bda9-74bebe8eaf1b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:47.757165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 731/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:49.950692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bc8c1-7f5d-44cf-b50f-30b9802358e1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:49.951231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bc8c1-7f5d-44cf-b50f-30b9802358e1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:49.951231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bc8c1-7f5d-44cf-b50f-30b9802358e1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:49.952611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:49.957961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a058-b0cb-4888-989e-ef953c723279 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:49.958370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a058-b0cb-4888-989e-ef953c723279 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:49.958494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a058-b0cb-4888-989e-ef953c723279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:50.005738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:50.022929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-36d4a058-b0cb-4888-989e-ef953c723279 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:50.024179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-36d4a058-b0cb-4888-989e-ef953c723279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:50.117557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:50.400909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40bc8c1-7f5d-44cf-b50f-30b9802358e1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:50.402435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a40bc8c1-7f5d-44cf-b50f-30b9802358e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:50.503989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40bc8c1-7f5d-44cf-b50f-30b9802358e1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:50.542869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:50.575896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040342ca-d4b6-46d7-aa72-027d756cd11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:50.576502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040342ca-d4b6-46d7-aa72-027d756cd11b service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:50.576782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040342ca-d4b6-46d7-aa72-027d756cd11b service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:50.577053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040342ca-d4b6-46d7-aa72-027d756cd11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:50.578680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-040342ca-d4b6-46d7-aa72-027d756cd11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:50.685821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040342ca-d4b6-46d7-aa72-027d756cd11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:50.687282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040342ca-d4b6-46d7-aa72-027d756cd11b service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:50.705434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 717/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:52.734424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650f974-304e-4c7e-8a54-47169ea23e50 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:52.734952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650f974-304e-4c7e-8a54-47169ea23e50 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:52.734952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650f974-304e-4c7e-8a54-47169ea23e50 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:52.736383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:52.741395 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53a9f-d7c6-4bba-8cc3-3dcccd3965df None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:52.741799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53a9f-d7c6-4bba-8cc3-3dcccd3965df None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:52.741940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53a9f-d7c6-4bba-8cc3-3dcccd3965df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:52.807963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1dd53a9f-d7c6-4bba-8cc3-3dcccd3965df None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:52.809159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1dd53a9f-d7c6-4bba-8cc3-3dcccd3965df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:52.830511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:52.904297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:53.201370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650f974-304e-4c7e-8a54-47169ea23e50 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:53.202910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e650f974-304e-4c7e-8a54-47169ea23e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:53.305280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650f974-304e-4c7e-8a54-47169ea23e50 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:53.324658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:53.380248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff999-bc2e-4a84-b962-3813ca0f8327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:53.380851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff999-bc2e-4a84-b962-3813ca0f8327 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:53.381129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff999-bc2e-4a84-b962-3813ca0f8327 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:53.381400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff999-bc2e-4a84-b962-3813ca0f8327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:53.382993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-926ff999-bc2e-4a84-b962-3813ca0f8327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:53.491019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff999-bc2e-4a84-b962-3813ca0f8327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:53.492391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff999-bc2e-4a84-b962-3813ca0f8327 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:53.510131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 734/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:55.476592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb2674-f5ba-4e81-bad4-5ecfe6f72aca None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:55.477135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb2674-f5ba-4e81-bad4-5ecfe6f72aca None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:55.477135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb2674-f5ba-4e81-bad4-5ecfe6f72aca None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:55.478497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:55.483998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aadd9-f0b1-45f5-b22a-8d288de2977c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:55.484457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aadd9-f0b1-45f5-b22a-8d288de2977c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:55.484574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aadd9-f0b1-45f5-b22a-8d288de2977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:55.548085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:55.551101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7a8aadd9-f0b1-45f5-b22a-8d288de2977c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:55.552660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a8aadd9-f0b1-45f5-b22a-8d288de2977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:55.647115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:55.928695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb2674-f5ba-4e81-bad4-5ecfe6f72aca None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:55.930279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e7bb2674-f5ba-4e81-bad4-5ecfe6f72aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:56.032631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb2674-f5ba-4e81-bad4-5ecfe6f72aca None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:56.080438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:56.104018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff756c4-fc1a-4a73-a314-d2214622d36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:56.104618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff756c4-fc1a-4a73-a314-d2214622d36c service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:56.104891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff756c4-fc1a-4a73-a314-d2214622d36c service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:56.105164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff756c4-fc1a-4a73-a314-d2214622d36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:56.106736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7ff756c4-fc1a-4a73-a314-d2214622d36c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:56.211843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff756c4-fc1a-4a73-a314-d2214622d36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:56.213183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff756c4-fc1a-4a73-a314-d2214622d36c service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:56.230757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 720/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:05:58.200022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397190-fac4-4d9a-83b1-380d207c8c68 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:58.200575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397190-fac4-4d9a-83b1-380d207c8c68 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:58.200575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397190-fac4-4d9a-83b1-380d207c8c68 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:58.201934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:05:58.208019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c4cc-155d-43ca-b856-f152c4e2af80 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:58.208448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c4cc-155d-43ca-b856-f152c4e2af80 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:58.208588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c4cc-155d-43ca-b856-f152c4e2af80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:58.275434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4199c4cc-155d-43ca-b856-f152c4e2af80 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:05:58.276602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4199c4cc-155d-43ca-b856-f152c4e2af80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:58.305547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:58.372651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:05:58.658279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397190-fac4-4d9a-83b1-380d207c8c68 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:05:58.659905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b397190-fac4-4d9a-83b1-380d207c8c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:58.761626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397190-fac4-4d9a-83b1-380d207c8c68 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:05:58.789496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:05:58.834648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f2105-e47f-4019-bc01-fc2de2b277c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:05:58.835167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f2105-e47f-4019-bc01-fc2de2b277c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:05:58.835444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f2105-e47f-4019-bc01-fc2de2b277c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:05:58.835713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f2105-e47f-4019-bc01-fc2de2b277c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:05:58.837327 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c86f2105-e47f-4019-bc01-fc2de2b277c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:05:58.944137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f2105-e47f-4019-bc01-fc2de2b277c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:05:58.945550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f2105-e47f-4019-bc01-fc2de2b277c9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:05:58.963472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 737/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:00.946560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c202-b794-4813-a789-4503e86060ef None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:00.947113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c202-b794-4813-a789-4503e86060ef None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:00.947113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c202-b794-4813-a789-4503e86060ef None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:00.948490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:00.953761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480422a8-86b7-4c5b-9d51-8621aeb9636e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:00.954141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480422a8-86b7-4c5b-9d51-8621aeb9636e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:00.954314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480422a8-86b7-4c5b-9d51-8621aeb9636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:01.018310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-480422a8-86b7-4c5b-9d51-8621aeb9636e None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:01.019530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-480422a8-86b7-4c5b-9d51-8621aeb9636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:01.027921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:01.116508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:01.395822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754c202-b794-4813-a789-4503e86060ef None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:01.398160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8754c202-b794-4813-a789-4503e86060ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:01.498670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754c202-b794-4813-a789-4503e86060ef None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:01.553347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:01.571727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4de323-0346-43b6-9d21-165ec5cc10c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:01.572363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de323-0346-43b6-9d21-165ec5cc10c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:01.572639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de323-0346-43b6-9d21-165ec5cc10c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:01.572907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de323-0346-43b6-9d21-165ec5cc10c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:01.574496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9a4de323-0346-43b6-9d21-165ec5cc10c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:01.685338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4de323-0346-43b6-9d21-165ec5cc10c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:01.686787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4de323-0346-43b6-9d21-165ec5cc10c9 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:01.705525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 723/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:03.682432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7217c-d0f1-4ad3-87d8-0c75d43bff89 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:03.683036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7217c-d0f1-4ad3-87d8-0c75d43bff89 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:03.683036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7217c-d0f1-4ad3-87d8-0c75d43bff89 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:03.684372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:03.690114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc438c-5536-4909-a2b6-4d6b50ce2d46 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:03.690536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc438c-5536-4909-a2b6-4d6b50ce2d46 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:03.690648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc438c-5536-4909-a2b6-4d6b50ce2d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:03.756927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0dfc438c-5536-4909-a2b6-4d6b50ce2d46 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:03.758127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0dfc438c-5536-4909-a2b6-4d6b50ce2d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:03.796035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:03.853788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:06:04.138388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7217c-d0f1-4ad3-87d8-0c75d43bff89 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:04.139975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-36c7217c-d0f1-4ad3-87d8-0c75d43bff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:04.243165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7217c-d0f1-4ad3-87d8-0c75d43bff89 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:04.279671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:04.316005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a6016-4c91-4ec5-876e-ce8391fdebd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:04.316609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a6016-4c91-4ec5-876e-ce8391fdebd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:04.316888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a6016-4c91-4ec5-876e-ce8391fdebd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:04.317175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a6016-4c91-4ec5-876e-ce8391fdebd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:04.318735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c7a6016-4c91-4ec5-876e-ce8391fdebd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:04.423683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a6016-4c91-4ec5-876e-ce8391fdebd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:04.425138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a6016-4c91-4ec5-876e-ce8391fdebd4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:04.444750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 740/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:06.354044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018c2a-040e-4d02-82dc-5dc27bb3562a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:06.354581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018c2a-040e-4d02-82dc-5dc27bb3562a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:06.354581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018c2a-040e-4d02-82dc-5dc27bb3562a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:06.355965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:06.361304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ce323-aa42-4bfa-a108-5459146e9641 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:06.361748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ce323-aa42-4bfa-a108-5459146e9641 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:06.361877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ce323-aa42-4bfa-a108-5459146e9641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:06.426515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f59ce323-aa42-4bfa-a108-5459146e9641 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:06.427729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f59ce323-aa42-4bfa-a108-5459146e9641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:06.445676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:06.522921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:06:06.808888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb018c2a-040e-4d02-82dc-5dc27bb3562a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:06.810421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eb018c2a-040e-4d02-82dc-5dc27bb3562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:06.912607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb018c2a-040e-4d02-82dc-5dc27bb3562a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:06.987015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:06.994329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f3fef-1bd8-42e8-acce-27f4fae798d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:06.994936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f3fef-1bd8-42e8-acce-27f4fae798d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:06.995216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f3fef-1bd8-42e8-acce-27f4fae798d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:06.995487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f3fef-1bd8-42e8-acce-27f4fae798d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:06.997114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6e2f3fef-1bd8-42e8-acce-27f4fae798d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:07.104281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f3fef-1bd8-42e8-acce-27f4fae798d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:07.105725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f3fef-1bd8-42e8-acce-27f4fae798d2 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:07.123499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 726/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:09.062561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262eb9c-0062-49d2-a159-5f2e2dd24e2e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:09.063153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262eb9c-0062-49d2-a159-5f2e2dd24e2e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:09.063153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262eb9c-0062-49d2-a159-5f2e2dd24e2e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:09.064499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:09.069943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6784b8b-1d1f-4106-93cd-20cb955b2cea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:09.070378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6784b8b-1d1f-4106-93cd-20cb955b2cea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:09.070504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6784b8b-1d1f-4106-93cd-20cb955b2cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:09.135737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a6784b8b-1d1f-4106-93cd-20cb955b2cea None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:09.136964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a6784b8b-1d1f-4106-93cd-20cb955b2cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:09.184474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:09.232735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:09.512016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262eb9c-0062-49d2-a159-5f2e2dd24e2e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:09.513543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a262eb9c-0062-49d2-a159-5f2e2dd24e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:09.614368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262eb9c-0062-49d2-a159-5f2e2dd24e2e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:09.660218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:09.685484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fce478-9cc0-48bf-b4b4-f9556df2fdae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:09.686080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fce478-9cc0-48bf-b4b4-f9556df2fdae service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:09.686355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fce478-9cc0-48bf-b4b4-f9556df2fdae service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:09.686621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fce478-9cc0-48bf-b4b4-f9556df2fdae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:09.688299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-26fce478-9cc0-48bf-b4b4-f9556df2fdae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:09.794768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fce478-9cc0-48bf-b4b4-f9556df2fdae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:09.796143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fce478-9cc0-48bf-b4b4-f9556df2fdae service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:09.814504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 743/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:19.230928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=22 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:19.811760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=66 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:29.181631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4beba3-91c5-4827-be00-91c2ecaaea18 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:29.182165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4beba3-91c5-4827-be00-91c2ecaaea18 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:29.182165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4beba3-91c5-4827-be00-91c2ecaaea18 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:29.183522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 728/1471] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:29.188676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8347-05a0-49f7-9a66-b36a4d0746ea None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:29.189101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8347-05a0-49f7-9a66-b36a4d0746ea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:29.189197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b8347-05a0-49f7-9a66-b36a4d0746ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:29.256717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c86b8347-05a0-49f7-9a66-b36a4d0746ea None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:29.257939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c86b8347-05a0-49f7-9a66-b36a4d0746ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:29.352113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 744/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:29.647572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568d06-8d9e-4d22-97c0-c458fa8bfd56 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:29.647888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568d06-8d9e-4d22-97c0-c458fa8bfd56 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:29.648163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568d06-8d9e-4d22-97c0-c458fa8bfd56 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:29.649534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 729/1473] 10.222.0.23 () {54 vars in 961 bytes} [Tue Sep 6 01:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:29.654361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86a25-3337-40a5-b37f-497026bd2af6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:29.654650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86a25-3337-40a5-b37f-497026bd2af6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:29.654921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86a25-3337-40a5-b37f-497026bd2af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:29.728725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b1a86a25-3337-40a5-b37f-497026bd2af6 None None] MFA Rules not processed for user `10bbe5bd5324482b9a74836a552a077d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:29.729927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b1a86a25-3337-40a5-b37f-497026bd2af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:29.750887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:29.805171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 745/1474] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Sep 6 01:06:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:29.813231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7568d06-8d9e-4d22-97c0-c458fa8bfd56 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:29.814731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a7568d06-8d9e-4d22-97c0-c458fa8bfd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:29.914617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7568d06-8d9e-4d22-97c0-c458fa8bfd56 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:29.989291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f42d-a899-474f-8277-cec6287e8846 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:29.989896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f42d-a899-474f-8277-cec6287e8846 service glance] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:29.990180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f42d-a899-474f-8277-cec6287e8846 service glance] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:29.990453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f42d-a899-474f-8277-cec6287e8846 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:29.992085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2893f42d-a899-474f-8277-cec6287e8846 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:30.162777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:30.217129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893f42d-a899-474f-8277-cec6287e8846 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:30.219090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893f42d-a899-474f-8277-cec6287e8846 service glance] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:30.237161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 730/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:32.265010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d6c85-bd48-4ab3-958d-3884f5203004 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:32.265601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d6c85-bd48-4ab3-958d-3884f5203004 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:32.265601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d6c85-bd48-4ab3-958d-3884f5203004 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:32.266921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 746/1476] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:32.272617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a12b0-430f-46b3-ade3-671051c7b868 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:32.273056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a12b0-430f-46b3-ade3-671051c7b868 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:32.273167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a12b0-430f-46b3-ade3-671051c7b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:32.341113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bd8a12b0-430f-46b3-ade3-671051c7b868 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:32.342327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bd8a12b0-430f-46b3-ade3-671051c7b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:32.435879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 731/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:32.713881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d6c85-bd48-4ab3-958d-3884f5203004 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:32.715412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d78d6c85-bd48-4ab3-958d-3884f5203004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:32.837438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d6c85-bd48-4ab3-958d-3884f5203004 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:32.885069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:32.930539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db1b7d-7419-4878-92fb-397292436b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:32.931136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db1b7d-7419-4878-92fb-397292436b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:32.931411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db1b7d-7419-4878-92fb-397292436b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:32.931678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db1b7d-7419-4878-92fb-397292436b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:32.932334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db1b7d-7419-4878-92fb-397292436b51 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:32.934032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-f3db1b7d-7419-4878-92fb-397292436b51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:06:32.934032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:06:32.934032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:06:32.935152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db1b7d-7419-4878-92fb-397292436b51 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:32.953857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 747/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 6 01:06:32 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:06:34.862825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfe3dd-5383-411c-8b5b-733802821d7a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:34.863346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfe3dd-5383-411c-8b5b-733802821d7a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:34.863346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfe3dd-5383-411c-8b5b-733802821d7a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:34.864662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 732/1479] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:34.869909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d159c76-f6ab-44c5-b32a-016da0007b9b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:34.870306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d159c76-f6ab-44c5-b32a-016da0007b9b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:34.870469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d159c76-f6ab-44c5-b32a-016da0007b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:34.935588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4d159c76-f6ab-44c5-b32a-016da0007b9b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:34.936830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4d159c76-f6ab-44c5-b32a-016da0007b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:35.031522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 748/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:35.321827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfe3dd-5383-411c-8b5b-733802821d7a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:35.323610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5ebfe3dd-5383-411c-8b5b-733802821d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:35.334239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:35.466443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfe3dd-5383-411c-8b5b-733802821d7a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:35.546529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:35.579718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81647875-1cf0-4820-bd43-a1e03a23dfc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:35.580280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81647875-1cf0-4820-bd43-a1e03a23dfc5 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:35.580560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81647875-1cf0-4820-bd43-a1e03a23dfc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:35.580858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81647875-1cf0-4820-bd43-a1e03a23dfc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:35.582486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-81647875-1cf0-4820-bd43-a1e03a23dfc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:35.669156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81647875-1cf0-4820-bd43-a1e03a23dfc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:35.670740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81647875-1cf0-4820-bd43-a1e03a23dfc5 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:35.689239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 733/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:37.785642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310c260-9b51-403a-b125-7c2d65826861 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:37.786191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310c260-9b51-403a-b125-7c2d65826861 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:37.786300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310c260-9b51-403a-b125-7c2d65826861 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:37.787667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 749/1482] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:37.793361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca93cc-b098-49d2-b8e9-fd075ed33204 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:37.793800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca93cc-b098-49d2-b8e9-fd075ed33204 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:37.793933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca93cc-b098-49d2-b8e9-fd075ed33204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:37.858597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1dca93cc-b098-49d2-b8e9-fd075ed33204 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:37.860294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1dca93cc-b098-49d2-b8e9-fd075ed33204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:37.905210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:37.964401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 734/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:38.258217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310c260-9b51-403a-b125-7c2d65826861 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:38.259787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9310c260-9b51-403a-b125-7c2d65826861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:38.283527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:38.362031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310c260-9b51-403a-b125-7c2d65826861 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:38.432672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e16357-e778-4c2f-a03b-ff73df380966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:38.433348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e16357-e778-4c2f-a03b-ff73df380966 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:38.433631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e16357-e778-4c2f-a03b-ff73df380966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:38.433897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e16357-e778-4c2f-a03b-ff73df380966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:38.435462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-94e16357-e778-4c2f-a03b-ff73df380966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:38.509883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:38.544406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e16357-e778-4c2f-a03b-ff73df380966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:38.545825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e16357-e778-4c2f-a03b-ff73df380966 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:38.565755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 750/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:40.550574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155934b-25f8-4537-871b-7c0134cd4d4a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:40.551101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155934b-25f8-4537-871b-7c0134cd4d4a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:40.551101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155934b-25f8-4537-871b-7c0134cd4d4a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:40.552453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 735/1485] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:40.557743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd6ff2-1036-4d82-a312-b966a5135291 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:40.558157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd6ff2-1036-4d82-a312-b966a5135291 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:40.558320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd6ff2-1036-4d82-a312-b966a5135291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:40.624020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1efd6ff2-1036-4d82-a312-b966a5135291 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:40.625220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1efd6ff2-1036-4d82-a312-b966a5135291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:40.721480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 751/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:41.016084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155934b-25f8-4537-871b-7c0134cd4d4a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:41.017634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a155934b-25f8-4537-871b-7c0134cd4d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:41.119080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155934b-25f8-4537-871b-7c0134cd4d4a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:41.187930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:41.189851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dad3f-b0a6-4107-a3db-ddc5f0d45300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:41.190513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dad3f-b0a6-4107-a3db-ddc5f0d45300 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:41.191140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dad3f-b0a6-4107-a3db-ddc5f0d45300 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:41.191589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dad3f-b0a6-4107-a3db-ddc5f0d45300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:41.193383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a67dad3f-b0a6-4107-a3db-ddc5f0d45300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:41.299919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dad3f-b0a6-4107-a3db-ddc5f0d45300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:41.301333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dad3f-b0a6-4107-a3db-ddc5f0d45300 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:41.319212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 736/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:43.294348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dd917-2dbb-4447-90cb-ef4aa0060e24 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:43.294907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dd917-2dbb-4447-90cb-ef4aa0060e24 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:43.294907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dd917-2dbb-4447-90cb-ef4aa0060e24 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:43.296316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 752/1488] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:43.301406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06343c6-b4ce-4ddf-9315-1cbcd10c54c1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:43.301847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06343c6-b4ce-4ddf-9315-1cbcd10c54c1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:43.302045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06343c6-b4ce-4ddf-9315-1cbcd10c54c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:43.381449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b06343c6-b4ce-4ddf-9315-1cbcd10c54c1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:43.382666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b06343c6-b4ce-4ddf-9315-1cbcd10c54c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:43.435051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:43.479074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 737/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:43.770439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dd917-2dbb-4447-90cb-ef4aa0060e24 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:43.772093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de7dd917-2dbb-4447-90cb-ef4aa0060e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:43.806854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:43.875610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dd917-2dbb-4447-90cb-ef4aa0060e24 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:43.947352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94688258-0ab8-4085-bc17-107b30a77bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:43.947978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94688258-0ab8-4085-bc17-107b30a77bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:43.948259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94688258-0ab8-4085-bc17-107b30a77bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:43.948525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94688258-0ab8-4085-bc17-107b30a77bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:43.950098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-94688258-0ab8-4085-bc17-107b30a77bd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:44.026613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:44.055500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94688258-0ab8-4085-bc17-107b30a77bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:44.057033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94688258-0ab8-4085-bc17-107b30a77bd3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:44.075358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 753/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:46.183342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5065-59d0-4760-bf5e-95f40b07dcf4 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:46.183905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5065-59d0-4760-bf5e-95f40b07dcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:46.183905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5065-59d0-4760-bf5e-95f40b07dcf4 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:46.185247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 738/1491] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 6 01:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:06:46.190416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbed58c-5efb-4ca4-beef-66f437490710 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:46.190851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbed58c-5efb-4ca4-beef-66f437490710 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:46.190989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbed58c-5efb-4ca4-beef-66f437490710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:46.256988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ccbed58c-5efb-4ca4-beef-66f437490710 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:06:46.258185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccbed58c-5efb-4ca4-beef-66f437490710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:46.352595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 754/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 6 01:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:06:46.658446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb5065-59d0-4760-bf5e-95f40b07dcf4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:06:46.660050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3fb5065-59d0-4760-bf5e-95f40b07dcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:46.762232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb5065-59d0-4760-bf5e-95f40b07dcf4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:06:46.834613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3473729-29f3-4bd2-8e5d-c38311ea8791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:06:46.835140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3473729-29f3-4bd2-8e5d-c38311ea8791 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:06:46.835419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3473729-29f3-4bd2-8e5d-c38311ea8791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:06:46.835689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3473729-29f3-4bd2-8e5d-c38311ea8791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:06:46.837288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f3473729-29f3-4bd2-8e5d-c38311ea8791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:06:46.846221 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:46.944037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3473729-29f3-4bd2-8e5d-c38311ea8791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:06:46.945474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3473729-29f3-4bd2-8e5d-c38311ea8791 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:06:46.964469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 739/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:06:56.350784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:06:56.962220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=50 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:05.873612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:05.874229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:05.874229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:05.950030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:08:05.951267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:06.046209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 755/1494] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:08:06.054092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ccfd7-dd63-4db2-98ff-ba8fa8dd5cbc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:06.054560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ccfd7-dd63-4db2-98ff-ba8fa8dd5cbc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:06.054654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ccfd7-dd63-4db2-98ff-ba8fa8dd5cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:06.127616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-334ccfd7-dd63-4db2-98ff-ba8fa8dd5cbc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:08:06.128770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-334ccfd7-dd63-4db2-98ff-ba8fa8dd5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:06.224022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 740/1495] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:08:06.228716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:06.229931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:06.241933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3e5f9-8a9c-4483-a7cc-4054a2318958 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:06.306312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:06.338780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:06.339392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:06.339669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:06.339968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:06.340612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:08:06.340915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:06.342456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:06.342456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:08:06.342456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:08:06.343340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:06.359957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 756/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:08:06.368543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ccfd7-dd63-4db2-98ff-ba8fa8dd5cbc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:06.377590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ccfd7-dd63-4db2-98ff-ba8fa8dd5cbc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:06.437748 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:06.471847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:06.472443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:06.472717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:06.472981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:06.473615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:06.475399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:06.475399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 06 01:08:06.475399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: Sep 06 01:08:06.476602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:06.531820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 741/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:08:06 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:08:06.535897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:06.537630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:06.570430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4913d4e-aa05-4eeb-8450-8b5a7c275c8d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:06.667220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:06.668105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:06.668388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:06.668658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:06.669293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:06.670330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:06.670330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:08:06.670330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:08:06.671535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:06.714338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 757/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:08:06 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:08:06.716431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:06.719468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:06.779675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947930bf-9775-4392-a94f-618eb828d60d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:06.860696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:06.871909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:06.872502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:06.873315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:06.873583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:06.874206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:06.875204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:06.875204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 06 01:08:06.875204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: Sep 06 01:08:06.876100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:06.888073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 742/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:08:06.892567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:06.894123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:06.905463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf01b5e-7e2d-4453-8f57-4ea408db930f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:06.930276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:06.999234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3ee7-1f90-4c47-a6ed-fd4cae6e2b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:06.999796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3ee7-1f90-4c47-a6ed-fd4cae6e2b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:07.000073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3ee7-1f90-4c47-a6ed-fd4cae6e2b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:07.000338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3ee7-1f90-4c47-a6ed-fd4cae6e2b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:07.000990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3ee7-1f90-4c47-a6ed-fd4cae6e2b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:07.002014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-02bc3ee7-1f90-4c47-a6ed-fd4cae6e2b23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:07.002014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:08:07.002014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:08:07.002914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3ee7-1f90-4c47-a6ed-fd4cae6e2b23 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:07.019209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 758/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:08:07.023484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:07.025058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:07.062232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb22b8c-48f1-45c0-921c-16eb02cbef4e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:07.158555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af514c6-8144-496a-8b9a-6301a100afec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:07.159203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af514c6-8144-496a-8b9a-6301a100afec admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:07.159477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af514c6-8144-496a-8b9a-6301a100afec admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:07.159742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af514c6-8144-496a-8b9a-6301a100afec admin admin] PATH_INFO: `/v3/projects/de9c3f045dec425a9d04bf534570d246/users/3c8a3cddf425447095bd0f20485aa915/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:07.181462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af514c6-8144-496a-8b9a-6301a100afec admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9c3f045dec425a9d04bf534570d246, user_id=3c8a3cddf425447095bd0f20485aa915, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:07.182524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-1af514c6-8144-496a-8b9a-6301a100afec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:07.182524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 06 01:08:07.182524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: Sep 06 01:08:07.184485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af514c6-8144-496a-8b9a-6301a100afec admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:07.209234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 743/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:08:07 2022] PUT /identity/v3/projects/de9c3f045dec425a9d04bf534570d246/users/3c8a3cddf425447095bd0f20485aa915/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:08:07.221121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4d23-38e5-4664-b371-41f63835c629 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:07.221411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4d23-38e5-4664-b371-41f63835c629 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:07.221683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4d23-38e5-4664-b371-41f63835c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:07.278581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cacc4d23-38e5-4664-b371-41f63835c629 None None] MFA Rules not processed for user `3c8a3cddf425447095bd0f20485aa915`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:08:07.279738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cacc4d23-38e5-4664-b371-41f63835c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:07.354964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 759/1502] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:08:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:08:07.368627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe074d2-741d-4cbb-9b90-433ae1b3b013 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:07.368921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe074d2-741d-4cbb-9b90-433ae1b3b013 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:07.369203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe074d2-741d-4cbb-9b90-433ae1b3b013 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:07.370567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 744/1503] 10.222.0.23 () {54 vars in 964 bytes} [Tue Sep 6 01:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:08:07.375907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f81c9-32c1-4a42-b5d3-9cfb8facf679 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:07.376195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f81c9-32c1-4a42-b5d3-9cfb8facf679 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:07.376468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f81c9-32c1-4a42-b5d3-9cfb8facf679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:07.407883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:07.453123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-210f81c9-32c1-4a42-b5d3-9cfb8facf679 None None] MFA Rules not processed for user `1c737d4ca9ea488eb0aad2b328de193c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:08:07.454326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-210f81c9-32c1-4a42-b5d3-9cfb8facf679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:07.527174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 760/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Sep 6 01:08:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:08:07.535546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe074d2-741d-4cbb-9b90-433ae1b3b013 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:07.537034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dbe074d2-741d-4cbb-9b90-433ae1b3b013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:07.586837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:07.640897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe074d2-741d-4cbb-9b90-433ae1b3b013 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:07.712468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:07.713066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:07.713343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:07.713611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:07.715190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:07.813359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:07.814739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:07.818170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:07.832595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 745/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:08:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:08:07.877794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb438250-4cfa-4ca2-83f8-39563445d3ad None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:07.878085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb438250-4cfa-4ca2-83f8-39563445d3ad None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:07.878362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb438250-4cfa-4ca2-83f8-39563445d3ad None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:07.879644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 761/1506] 10.222.0.23 () {52 vars in 830 bytes} [Tue Sep 6 01:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:08:07.883297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:07.884558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:07.986295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19526d43-b455-42ca-8b0d-ab3e58120d80 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:08.077101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:08.077731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:08.077908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:08.078173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] PATH_INFO: `/v3/users/3c8a3cddf425447095bd0f20485aa915` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:08.089865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8a3cddf425447095bd0f20485aa915)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:08.090877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:08.090877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 06 01:08:08.090877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: Sep 06 01:08:08.092130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:08.101406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:08.157043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8a3cddf425447095bd0f20485aa915'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:08.157483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8a3cddf425447095bd0f20485aa915'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:08.167326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8a3cddf425447095bd0f20485aa915'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:08.190375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8a3cddf425447095bd0f20485aa915'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:08.190902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8a3cddf425447095bd0f20485aa915'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:08.209315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 746/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:08:07 2022] DELETE /identity/v3/users/3c8a3cddf425447095bd0f20485aa915 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:08:08.213717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:08.214164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:08.214295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:08.270601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:08:08.271843 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:08.345136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:08.358788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 762/1508] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:08:08.368902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:08.370110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:08.469713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215a9e4-d046-4b49-968d-305ca7b3454c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:08.526669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:08.547292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720f121-30b0-4e15-a7ea-228a717828fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:08.547929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720f121-30b0-4e15-a7ea-228a717828fd service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:08.548209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720f121-30b0-4e15-a7ea-228a717828fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:08.548480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720f121-30b0-4e15-a7ea-228a717828fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:08.550067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8720f121-30b0-4e15-a7ea-228a717828fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:08.657322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720f121-30b0-4e15-a7ea-228a717828fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:08.658692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720f121-30b0-4e15-a7ea-228a717828fd service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:08.677397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 747/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:08:09.021595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:08:09.022850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:08:09.034589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9553a3-f853-4a95-8347-dcc0d204040c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:08:09.131460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:08:09.132120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:08:09.132398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:08:09.132670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] PATH_INFO: `/v3/projects/de9c3f045dec425a9d04bf534570d246` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:08:09.142280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] RBAC: Authorizing `identity:delete_project(project_id=de9c3f045dec425a9d04bf534570d246)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:08:09.143294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:08:09.143294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:08:09.143294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:08:09.144521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:08:09.201587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de9c3f045dec425a9d04bf534570d246', 'request_id': 'req-c98fa188-b148-4f8a-8f19-eb536fb4231a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:09.207414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de9c3f045dec425a9d04bf534570d246', 'request_id': 'req-c98fa188-b148-4f8a-8f19-eb536fb4231a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:09.216199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de9c3f045dec425a9d04bf534570d246', 'request_id': 'req-c98fa188-b148-4f8a-8f19-eb536fb4231a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:09.250893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] The token cache is being invalidate because project de9c3f045dec425a9d04bf534570d246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:08:09.251183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:09.251715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c98fa188-b148-4f8a-8f19-eb536fb4231a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:08:09.253075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 763/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:08:09 2022] DELETE /identity/v3/projects/de9c3f045dec425a9d04bf534570d246 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:08:18.675123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=62 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:08:19.248719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:09:31.862939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939eac-c580-4b38-9d91-ea7a768adfb8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:31.863537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939eac-c580-4b38-9d91-ea7a768adfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:31.863537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939eac-c580-4b38-9d91-ea7a768adfb8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:31.864905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:31.869761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada4ba-d3e9-4652-b340-bdbcb0381cc7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:31.870185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada4ba-d3e9-4652-b340-bdbcb0381cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:31.870330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada4ba-d3e9-4652-b340-bdbcb0381cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:31.949716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-73ada4ba-d3e9-4652-b340-bdbcb0381cc7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:31.950904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73ada4ba-d3e9-4652-b340-bdbcb0381cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:32.056563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:09:32.213112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939eac-c580-4b38-9d91-ea7a768adfb8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:32.215227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-45939eac-c580-4b38-9d91-ea7a768adfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:32.331362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939eac-c580-4b38-9d91-ea7a768adfb8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:32.336857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9820de3-a523-4758-85ca-ead3c466d2c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:32.337462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9820de3-a523-4758-85ca-ead3c466d2c5 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:32.337743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9820de3-a523-4758-85ca-ead3c466d2c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:32.338017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9820de3-a523-4758-85ca-ead3c466d2c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:32.346513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9820de3-a523-4758-85ca-ead3c466d2c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:32.348122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9820de3-a523-4758-85ca-ead3c466d2c5 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:32.357898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 749/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:33.200276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d950e-434d-4273-b5ab-aee7a7bded3f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:33.200791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d950e-434d-4273-b5ab-aee7a7bded3f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:33.200910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d950e-434d-4273-b5ab-aee7a7bded3f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:33.202206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:33.207052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902e4e8-503b-4cfb-9cf4-b4de9df5ed0b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:33.207496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902e4e8-503b-4cfb-9cf4-b4de9df5ed0b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:33.207624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902e4e8-503b-4cfb-9cf4-b4de9df5ed0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:33.227636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1902e4e8-503b-4cfb-9cf4-b4de9df5ed0b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:33.228840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1902e4e8-503b-4cfb-9cf4-b4de9df5ed0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:33.237161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:33.386338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d950e-434d-4273-b5ab-aee7a7bded3f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:33.401066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d950e-434d-4273-b5ab-aee7a7bded3f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:33.407172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef34982-2e2a-453a-b671-f4c31e7e845d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:33.407792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef34982-2e2a-453a-b671-f4c31e7e845d service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:33.408080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef34982-2e2a-453a-b671-f4c31e7e845d service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:33.408356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef34982-2e2a-453a-b671-f4c31e7e845d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:33.416429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef34982-2e2a-453a-b671-f4c31e7e845d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:33.417945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef34982-2e2a-453a-b671-f4c31e7e845d service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:33.427577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 766/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:34.381288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73e361-bb05-4f31-81fc-54db57cbb063 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:34.381855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73e361-bb05-4f31-81fc-54db57cbb063 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:34.381855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73e361-bb05-4f31-81fc-54db57cbb063 None None] PATH_INFO: `/v3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:34.383096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 751/1517] 10.222.0.26 () {54 vars in 925 bytes} [Tue Sep 6 01:09:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:09:34.389359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459877a-4815-46cc-a981-22bcb15badb6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:34.389798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459877a-4815-46cc-a981-22bcb15badb6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:34.389914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459877a-4815-46cc-a981-22bcb15badb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:34.448622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2459877a-4815-46cc-a981-22bcb15badb6 None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:34.449802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2459877a-4815-46cc-a981-22bcb15badb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:34.466408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:09:34.519435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 767/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 6 01:09:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:38.243965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93febf1-31c2-47cc-92c4-231ba1732a33 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:38.244474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93febf1-31c2-47cc-92c4-231ba1732a33 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:38.244583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93febf1-31c2-47cc-92c4-231ba1732a33 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:38.245835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 752/1519] 10.222.0.23 () {54 vars in 961 bytes} [Tue Sep 6 01:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:38.250839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbbf2f-0036-48c2-86e4-c9c2121a0d96 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:38.251318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbbf2f-0036-48c2-86e4-c9c2121a0d96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:38.251431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbbf2f-0036-48c2-86e4-c9c2121a0d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:38.270368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fbfbbf2f-0036-48c2-86e4-c9c2121a0d96 None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:38.271561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fbfbbf2f-0036-48c2-86e4-c9c2121a0d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:38.280562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 768/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Sep 6 01:09:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:38.287810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93febf1-31c2-47cc-92c4-231ba1732a33 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:38.301810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93febf1-31c2-47cc-92c4-231ba1732a33 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:38.306655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f6cd8-fb07-4bcb-aa54-196fa633da74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:38.307228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6cd8-fb07-4bcb-aa54-196fa633da74 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:38.307503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6cd8-fb07-4bcb-aa54-196fa633da74 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:38.307797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6cd8-fb07-4bcb-aa54-196fa633da74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:38.315684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f6cd8-fb07-4bcb-aa54-196fa633da74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:38.317271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f6cd8-fb07-4bcb-aa54-196fa633da74 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:38.326555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 753/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:44.677864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c2b6c-b3ce-4072-a18e-f9448d42b520 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:44.678431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c2b6c-b3ce-4072-a18e-f9448d42b520 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:44.678431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c2b6c-b3ce-4072-a18e-f9448d42b520 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:44.679735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:44.684437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6871d-1810-461a-9907-99aff1539e76 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:44.684889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6871d-1810-461a-9907-99aff1539e76 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:44.685003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6871d-1810-461a-9907-99aff1539e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:44.704451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-84b6871d-1810-461a-9907-99aff1539e76 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:44.705626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-84b6871d-1810-461a-9907-99aff1539e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:44.713276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:44.864566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c2b6c-b3ce-4072-a18e-f9448d42b520 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:44.879043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c2b6c-b3ce-4072-a18e-f9448d42b520 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:44.884323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0d991-cecf-42de-90e9-1d8c1a4fa2cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:44.884909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d991-cecf-42de-90e9-1d8c1a4fa2cf service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:44.885184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d991-cecf-42de-90e9-1d8c1a4fa2cf service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:44.885454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d991-cecf-42de-90e9-1d8c1a4fa2cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:44.893704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0d991-cecf-42de-90e9-1d8c1a4fa2cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:44.895247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0d991-cecf-42de-90e9-1d8c1a4fa2cf service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:44.904915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 770/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:45.711159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eac2dc-29c8-4b20-a1f3-42afaad6555f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:45.711705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eac2dc-29c8-4b20-a1f3-42afaad6555f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:45.711705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eac2dc-29c8-4b20-a1f3-42afaad6555f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:45.713116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 01:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:45.718479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea023ec-cbd6-4c15-9e53-0c70ade1a220 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:45.718914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea023ec-cbd6-4c15-9e53-0c70ade1a220 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:45.719054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea023ec-cbd6-4c15-9e53-0c70ade1a220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:45.738667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9ea023ec-cbd6-4c15-9e53-0c70ade1a220 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:45.739891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9ea023ec-cbd6-4c15-9e53-0c70ade1a220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:45.747561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 01:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:09:45.896050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eac2dc-29c8-4b20-a1f3-42afaad6555f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:45.910871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eac2dc-29c8-4b20-a1f3-42afaad6555f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:45.916229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680908c9-75df-4c85-a539-5195f7bd615d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:45.916818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680908c9-75df-4c85-a539-5195f7bd615d service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:45.917093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680908c9-75df-4c85-a539-5195f7bd615d service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:45.917359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680908c9-75df-4c85-a539-5195f7bd615d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:45.926053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680908c9-75df-4c85-a539-5195f7bd615d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:45.927608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680908c9-75df-4c85-a539-5195f7bd615d service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:45.936697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 756/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:49.804961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a341c-c9e5-49df-971b-276ef99e5bf0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:49.805526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a341c-c9e5-49df-971b-276ef99e5bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:49.805526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a341c-c9e5-49df-971b-276ef99e5bf0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:49.806853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:49.812511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134f7a-f247-4841-a5e4-02a7e19910f1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:49.812929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134f7a-f247-4841-a5e4-02a7e19910f1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:49.813070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134f7a-f247-4841-a5e4-02a7e19910f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:49.832612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-42134f7a-f247-4841-a5e4-02a7e19910f1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:49.833807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-42134f7a-f247-4841-a5e4-02a7e19910f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:49.841624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:50.127659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a341c-c9e5-49df-971b-276ef99e5bf0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:50.129398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-656a341c-c9e5-49df-971b-276ef99e5bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:50.210110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a341c-c9e5-49df-971b-276ef99e5bf0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:50.214118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250137d-890c-4ad2-b5b5-88937199cbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:50.214737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250137d-890c-4ad2-b5b5-88937199cbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:50.215017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250137d-890c-4ad2-b5b5-88937199cbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:50.215798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250137d-890c-4ad2-b5b5-88937199cbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:50.224724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250137d-890c-4ad2-b5b5-88937199cbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:50.226137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250137d-890c-4ad2-b5b5-88937199cbb9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:50.235929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 773/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:52.115457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2a827-9ed0-4cea-816d-5fc44a9488cf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:52.116065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2a827-9ed0-4cea-816d-5fc44a9488cf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:52.116065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2a827-9ed0-4cea-816d-5fc44a9488cf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:52.117363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:52.122600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7405-1829-4f03-86c6-85ea657318ad None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:52.122982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7405-1829-4f03-86c6-85ea657318ad None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:52.123150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb7405-1829-4f03-86c6-85ea657318ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:52.142631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f8fb7405-1829-4f03-86c6-85ea657318ad None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:52.143852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f8fb7405-1829-4f03-86c6-85ea657318ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:52.151501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:52.438325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2a827-9ed0-4cea-816d-5fc44a9488cf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:52.440093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bcf2a827-9ed0-4cea-816d-5fc44a9488cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:52.453481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2a827-9ed0-4cea-816d-5fc44a9488cf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:52.457036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397f45-47c4-4220-be40-09dbce22e6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:52.457625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397f45-47c4-4220-be40-09dbce22e6fe service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:52.457907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397f45-47c4-4220-be40-09dbce22e6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:52.458183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397f45-47c4-4220-be40-09dbce22e6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:52.467582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8397f45-47c4-4220-be40-09dbce22e6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:52.469031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8397f45-47c4-4220-be40-09dbce22e6fe service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:52.478491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 759/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:55.807471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d11d1-36a1-43e5-a51f-e5469f031522 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:55.808056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d11d1-36a1-43e5-a51f-e5469f031522 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:55.808056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d11d1-36a1-43e5-a51f-e5469f031522 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:55.809924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:55.815241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1bb0-b666-4614-bdbf-878cfacdcb2b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:55.815656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1bb0-b666-4614-bdbf-878cfacdcb2b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:55.815820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1bb0-b666-4614-bdbf-878cfacdcb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:55.835593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-028d1bb0-b666-4614-bdbf-878cfacdcb2b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:55.836808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-028d1bb0-b666-4614-bdbf-878cfacdcb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:55.844525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:56.386325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d11d1-36a1-43e5-a51f-e5469f031522 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:56.388210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a4d11d1-36a1-43e5-a51f-e5469f031522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:56.453986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:09:56.468126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d11d1-36a1-43e5-a51f-e5469f031522 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:56.471675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432f1a3-fa37-4edf-9a53-8b40d8d2ccc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:56.472332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432f1a3-fa37-4edf-9a53-8b40d8d2ccc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:56.472607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432f1a3-fa37-4edf-9a53-8b40d8d2ccc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:56.472874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432f1a3-fa37-4edf-9a53-8b40d8d2ccc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:56.482207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432f1a3-fa37-4edf-9a53-8b40d8d2ccc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:56.483641 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432f1a3-fa37-4edf-9a53-8b40d8d2ccc9 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:56.492888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 776/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:09:58.430688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6c36-a78f-4b64-aabb-21eabe0fc107 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:58.431251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6c36-a78f-4b64-aabb-21eabe0fc107 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:58.431251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6c36-a78f-4b64-aabb-21eabe0fc107 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:58.432655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 01:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:09:58.438827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f631b-7958-44b2-a7a9-e66900db355f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:58.439268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f631b-7958-44b2-a7a9-e66900db355f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:58.439537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f631b-7958-44b2-a7a9-e66900db355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:58.459493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-360f631b-7958-44b2-a7a9-e66900db355f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:09:58.460750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-360f631b-7958-44b2-a7a9-e66900db355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:58.469042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 01:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:09:59.017748 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b6c36-a78f-4b64-aabb-21eabe0fc107 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:09:59.019413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e38b6c36-a78f-4b64-aabb-21eabe0fc107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:09:59.026967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=84 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:09:59.034118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b6c36-a78f-4b64-aabb-21eabe0fc107 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:09:59.038071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906776d-10e0-4d3c-baf8-4e35c7b1adcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:09:59.038720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906776d-10e0-4d3c-baf8-4e35c7b1adcc service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:09:59.039087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906776d-10e0-4d3c-baf8-4e35c7b1adcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:09:59.039423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906776d-10e0-4d3c-baf8-4e35c7b1adcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:09:59.048479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906776d-10e0-4d3c-baf8-4e35c7b1adcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:09:59.049941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906776d-10e0-4d3c-baf8-4e35c7b1adcc service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:09:59.058994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 762/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:10:08.454787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=11 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:10:09.054965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:20.952910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5648ee6-6a62-429c-8ada-e0fc59697265 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:20.954064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5648ee6-6a62-429c-8ada-e0fc59697265 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:20.954576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5648ee6-6a62-429c-8ada-e0fc59697265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:20.985006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e5648ee6-6a62-429c-8ada-e0fc59697265 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:20.986611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e5648ee6-6a62-429c-8ada-e0fc59697265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:20.998644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 778/1540] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.008709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83111586-5634-4435-9ba8-7cf6aa9c226b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.009831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83111586-5634-4435-9ba8-7cf6aa9c226b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.010344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83111586-5634-4435-9ba8-7cf6aa9c226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.038646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab5946-e1cb-4a26-a6ca-5682440f9b80 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.038930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab5946-e1cb-4a26-a6ca-5682440f9b80 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.039385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab5946-e1cb-4a26-a6ca-5682440f9b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.041860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-83111586-5634-4435-9ba8-7cf6aa9c226b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.043096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-83111586-5634-4435-9ba8-7cf6aa9c226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.060025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 763/1541] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.064344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83111586-5634-4435-9ba8-7cf6aa9c226b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.072211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-17ab5946-e1cb-4a26-a6ca-5682440f9b80 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.073864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-17ab5946-e1cb-4a26-a6ca-5682440f9b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.084663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83111586-5634-4435-9ba8-7cf6aa9c226b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.093677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 779/1542] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:11:21.095352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.096239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.096708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.097195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.098134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:21.098638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.099949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:11:21.099949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 06 01:11:21.099949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: Sep 06 01:11:21.101556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.103265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fcdb0-eba5-4192-bf61-2ff265f1453a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.104535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fcdb0-eba5-4192-bf61-2ff265f1453a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.104987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fcdb0-eba5-4192-bf61-2ff265f1453a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.122294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 764/1543] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:11:21.131149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.139778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-807fcdb0-eba5-4192-bf61-2ff265f1453a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.139778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-807fcdb0-eba5-4192-bf61-2ff265f1453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.150547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fef6f8-12f7-416f-8ee5-bb8c64f05176 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.160200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 780/1544] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.164543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.164760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fcdb0-eba5-4192-bf61-2ff265f1453a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.165174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.165455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.165730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.166915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.169147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.175021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fcdb0-eba5-4192-bf61-2ff265f1453a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.194878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.195822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.196110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.196389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.197047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:21.197557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.199179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.224172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 781/1545] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:11:21.228116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.240860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee70099-cf30-4c39-bbf0-e9b94eb0e722 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.257852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ec09a-214d-4567-b8f7-357c7c6072b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.258476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec09a-214d-4567-b8f7-357c7c6072b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.258760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec09a-214d-4567-b8f7-357c7c6072b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.259033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec09a-214d-4567-b8f7-357c7c6072b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.259670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ec09a-214d-4567-b8f7-357c7c6072b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.260774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-053ec09a-214d-4567-b8f7-357c7c6072b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:11:21.260774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:11:21.260774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:11:21.261968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ec09a-214d-4567-b8f7-357c7c6072b7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.265512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 765/1546] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/projects => generated 389 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:21.269876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.322122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 782/1547] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/projects => generated 367 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:11:21.323532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f893b5-00c3-4cfd-86eb-b8a4e333d575 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.323871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f893b5-00c3-4cfd-86eb-b8a4e333d575 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.324128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f893b5-00c3-4cfd-86eb-b8a4e333d575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.364803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e9f893b5-00c3-4cfd-86eb-b8a4e333d575 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.366031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e9f893b5-00c3-4cfd-86eb-b8a4e333d575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.446613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc6327-eb11-4e9d-87be-39b008734dfb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.469309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 783/1548] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.470757 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f893b5-00c3-4cfd-86eb-b8a4e333d575 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.483091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f893b5-00c3-4cfd-86eb-b8a4e333d575 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.495093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b85aa-ee72-401e-b067-569a1e47b3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.496072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b85aa-ee72-401e-b067-569a1e47b3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.496602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b85aa-ee72-401e-b067-569a1e47b3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.497220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b85aa-ee72-401e-b067-569a1e47b3d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.498075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b85aa-ee72-401e-b067-569a1e47b3d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.500499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b85aa-ee72-401e-b067-569a1e47b3d6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.545054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 784/1549] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/users => generated 284 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:21.546846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8befcb-1216-49ad-87a5-e729522ce4f4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.547354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8befcb-1216-49ad-87a5-e729522ce4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.547873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8befcb-1216-49ad-87a5-e729522ce4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.571268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4a8befcb-1216-49ad-87a5-e729522ce4f4 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.573442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4a8befcb-1216-49ad-87a5-e729522ce4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.581795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d8711-a397-4e3c-8c42-7fcb1a67d942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.581795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8711-a397-4e3c-8c42-7fcb1a67d942 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.581795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8711-a397-4e3c-8c42-7fcb1a67d942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.581795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8711-a397-4e3c-8c42-7fcb1a67d942 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.582219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d8711-a397-4e3c-8c42-7fcb1a67d942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.582567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-681d8711-a397-4e3c-8c42-7fcb1a67d942 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:11:21.582567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 06 01:11:21.582567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: Sep 06 01:11:21.583791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d8711-a397-4e3c-8c42-7fcb1a67d942 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.591205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 785/1550] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.592721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac0441-ff97-476f-94d5-2ae6bbbb5726 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.593354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac0441-ff97-476f-94d5-2ae6bbbb5726 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.594080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac0441-ff97-476f-94d5-2ae6bbbb5726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.624311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 766/1551] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/users => generated 295 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:11:21.628224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632879-03e7-40ff-948a-630d28d01095 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.628224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632879-03e7-40ff-948a-630d28d01095 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.628224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632879-03e7-40ff-948a-630d28d01095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.629783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-87ac0441-ff97-476f-94d5-2ae6bbbb5726 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.631099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-87ac0441-ff97-476f-94d5-2ae6bbbb5726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.640662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 786/1552] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.642021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755633cd-0e95-4390-91c6-c7ee51c19cbe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.642305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755633cd-0e95-4390-91c6-c7ee51c19cbe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.642581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755633cd-0e95-4390-91c6-c7ee51c19cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.654180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3632879-03e7-40ff-948a-630d28d01095 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.657297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3632879-03e7-40ff-948a-630d28d01095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.667072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 767/1553] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.667254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-755633cd-0e95-4390-91c6-c7ee51c19cbe None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.670008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632879-03e7-40ff-948a-630d28d01095 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.670146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-755633cd-0e95-4390-91c6-c7ee51c19cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.678656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 787/1554] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.679983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178bb0d-e63c-4fcc-908f-8fecdad3aed7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.680492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178bb0d-e63c-4fcc-908f-8fecdad3aed7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.680625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632879-03e7-40ff-948a-630d28d01095 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.680819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178bb0d-e63c-4fcc-908f-8fecdad3aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.685375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.685978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.686255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.686524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.687160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.688843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.703366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 768/1555] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:21.703366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.704118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8178bb0d-e63c-4fcc-908f-8fecdad3aed7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.705329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8178bb0d-e63c-4fcc-908f-8fecdad3aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.714238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bdfe8-cdbb-4d6d-a1ed-55908dfccb8c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.719891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 788/1556] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.720731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178bb0d-e63c-4fcc-908f-8fecdad3aed7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.724490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83450018-ece3-4c2e-be58-c8481ec419dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.725098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450018-ece3-4c2e-be58-c8481ec419dd admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.725379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450018-ece3-4c2e-be58-c8481ec419dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.725637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450018-ece3-4c2e-be58-c8481ec419dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.726271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83450018-ece3-4c2e-be58-c8481ec419dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.727964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83450018-ece3-4c2e-be58-c8481ec419dd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.735268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178bb0d-e63c-4fcc-908f-8fecdad3aed7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.739218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 769/1557] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:21.740047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a884551-2510-4307-a8fa-97026d61fbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.742603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440d04d-7bd9-448b-b21f-bc006bd665c9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.742742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a884551-2510-4307-a8fa-97026d61fbae admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.742854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440d04d-7bd9-448b-b21f-bc006bd665c9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.742955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a884551-2510-4307-a8fa-97026d61fbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.743053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440d04d-7bd9-448b-b21f-bc006bd665c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.743162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a884551-2510-4307-a8fa-97026d61fbae admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.743162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a884551-2510-4307-a8fa-97026d61fbae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:21.743162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a884551-2510-4307-a8fa-97026d61fbae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.743851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a884551-2510-4307-a8fa-97026d61fbae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.762964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 789/1558] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:21.762964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67dc89-faf5-4005-9ec2-90711d03f464 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.762964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67dc89-faf5-4005-9ec2-90711d03f464 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.762964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67dc89-faf5-4005-9ec2-90711d03f464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.763442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f440d04d-7bd9-448b-b21f-bc006bd665c9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.763442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f440d04d-7bd9-448b-b21f-bc006bd665c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.772177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 770/1559] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.772177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f4255-ee37-43b7-a2b4-a39c5c3599bf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.772177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f4255-ee37-43b7-a2b4-a39c5c3599bf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.772177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f4255-ee37-43b7-a2b4-a39c5c3599bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.778607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ce67dc89-faf5-4005-9ec2-90711d03f464 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.779928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce67dc89-faf5-4005-9ec2-90711d03f464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.787712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 790/1560] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.788631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67dc89-faf5-4005-9ec2-90711d03f464 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.794467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-694f4255-ee37-43b7-a2b4-a39c5c3599bf None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.795814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-694f4255-ee37-43b7-a2b4-a39c5c3599bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.798178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67dc89-faf5-4005-9ec2-90711d03f464 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.805192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.806036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.806311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.806581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.807223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.809195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.809514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 771/1561] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.810862 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c95af-1885-4773-bd59-20d86536d276 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.811186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c95af-1885-4773-bd59-20d86536d276 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.811465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c95af-1885-4773-bd59-20d86536d276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.819928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:21.829144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 791/1562] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:21.829144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.832892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-164c95af-1885-4773-bd59-20d86536d276 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.834079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-164c95af-1885-4773-bd59-20d86536d276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.842938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 772/1563] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.843715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c95af-1885-4773-bd59-20d86536d276 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.844023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e9789-1aaf-4ee0-b080-419dadb3240e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.849082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.849939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.850349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.850750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.851705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:21.852202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.854824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.857417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c95af-1885-4773-bd59-20d86536d276 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.862078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c68da-76d6-4d63-bb5e-f5f81d3a0268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.862698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c68da-76d6-4d63-bb5e-f5f81d3a0268 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.863014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c68da-76d6-4d63-bb5e-f5f81d3a0268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.863287 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c68da-76d6-4d63-bb5e-f5f81d3a0268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.863957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c68da-76d6-4d63-bb5e-f5f81d3a0268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.865588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c68da-76d6-4d63-bb5e-f5f81d3a0268 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.869643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 792/1564] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:21.870488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.878708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 773/1565] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:21.878708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb34ff-d216-4b59-92d7-188727c7a570 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.878708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb34ff-d216-4b59-92d7-188727c7a570 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.878708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb34ff-d216-4b59-92d7-188727c7a570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.884479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef28386-6867-48fe-955b-4ea04290357e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.889563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.890159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.890440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.890711 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.891342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.893255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.896712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a8cb34ff-d216-4b59-92d7-188727c7a570 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:21.897904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a8cb34ff-d216-4b59-92d7-188727c7a570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:21.906343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 774/1566] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:21.910016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb34ff-d216-4b59-92d7-188727c7a570 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.910209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:21.920058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb34ff-d216-4b59-92d7-188727c7a570 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.925957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 793/1567] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:21.925957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.928219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.928819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.929099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.929371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.930040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:21.930346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.932007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:21.944158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 775/1568] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:21.944915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:21.960908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e01993-33c6-40e0-843b-2585851b4f9a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.988829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030ae04-c7bd-4ea9-aaf9-27fa3eaf5478 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:21.993446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ca70d-fafa-4aac-9826-5470a54c2c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:21.994049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca70d-fafa-4aac-9826-5470a54c2c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:21.994361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca70d-fafa-4aac-9826-5470a54c2c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:21.994633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca70d-fafa-4aac-9826-5470a54c2c3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:21.995269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ca70d-fafa-4aac-9826-5470a54c2c3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:21.995569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ca70d-fafa-4aac-9826-5470a54c2c3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:21.997304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ca70d-fafa-4aac-9826-5470a54c2c3b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.008675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 794/1569] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:22.010359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cf1e7-6440-4434-a38d-a8b0e33f3907 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.010640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cf1e7-6440-4434-a38d-a8b0e33f3907 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.010914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cf1e7-6440-4434-a38d-a8b0e33f3907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.025397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.026581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.026856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.027132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] PATH_INFO: `/v3/projects/b8d80cdbf2a642f7a5613a41e45fcb49/users/540fc07c6373415fb314c8af5010a936/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.030964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2a3cf1e7-6440-4434-a38d-a8b0e33f3907 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:22.032243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2a3cf1e7-6440-4434-a38d-a8b0e33f3907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:22.040564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 795/1570] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:22.041142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cf1e7-6440-4434-a38d-a8b0e33f3907 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.045549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d80cdbf2a642f7a5613a41e45fcb49, user_id=540fc07c6373415fb314c8af5010a936, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.048259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.051201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cf1e7-6440-4434-a38d-a8b0e33f3907 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.055854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.056506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.057294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.057571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] PATH_INFO: `/v3/projects/1c096a50088a4b8dbd00458b324f301e/users/2dc5c46544a447159b7fd3d225cd9697/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.068993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 776/1571] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:21 2022] PUT /identity/v3/projects/b8d80cdbf2a642f7a5613a41e45fcb49/users/540fc07c6373415fb314c8af5010a936/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:22.069796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.074111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c096a50088a4b8dbd00458b324f301e, user_id=2dc5c46544a447159b7fd3d225cd9697, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.075678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:11:22.075678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:11:22.075678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:11:22.077928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.092406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 796/1572] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:22 2022] PUT /identity/v3/projects/1c096a50088a4b8dbd00458b324f301e/users/2dc5c46544a447159b7fd3d225cd9697/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:22.093326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.133057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29a192-ef41-409c-b924-b1772b11288e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.154460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebe2f8-abc0-476a-a305-6acbd43f89dc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.159217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.159849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.160131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.160401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.161034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:22.161331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.162919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.175052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 797/1573] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:22.175995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.191651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:22.192120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3c9c4-db6e-4ce2-b58f-0f6abd198014 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.198861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.199604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.199818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.199986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.200269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.200556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.200910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.201061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.201348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.202348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.202836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.204981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.229461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 798/1574] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/users => generated 296 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.229461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.234125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 777/1575] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.234939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.239692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedd17a-0add-4672-9921-a200f3449373 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.273697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:22.298679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137569d-04de-409e-b6cf-33faf2999dd5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.299509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.300164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.300440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.300707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.301323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.303202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.303733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3fe6e-17a0-4b23-81ff-3bac387b5228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.304353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3fe6e-17a0-4b23-81ff-3bac387b5228 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.304630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3fe6e-17a0-4b23-81ff-3bac387b5228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.304897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3fe6e-17a0-4b23-81ff-3bac387b5228 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.305514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3fe6e-17a0-4b23-81ff-3bac387b5228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.307913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3fe6e-17a0-4b23-81ff-3bac387b5228 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.331400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 799/1576] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.332311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.337520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 778/1577] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.338800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529822-4c5a-4587-829a-54543db60a25 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.339081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529822-4c5a-4587-829a-54543db60a25 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.339353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529822-4c5a-4587-829a-54543db60a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.365405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cb529822-4c5a-4587-829a-54543db60a25 None None] MFA Rules not processed for user `540fc07c6373415fb314c8af5010a936`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:22.366583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cb529822-4c5a-4587-829a-54543db60a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:22.398493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8a7d8-3efd-4466-bcec-1e724ba84b5a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.430521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 779/1578] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:22.432508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced2d79-14bb-4a99-8909-a9950af4e854 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.432850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced2d79-14bb-4a99-8909-a9950af4e854 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.433179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced2d79-14bb-4a99-8909-a9950af4e854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.452789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9ced2d79-14bb-4a99-8909-a9950af4e854 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:22.453978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9ced2d79-14bb-4a99-8909-a9950af4e854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:22.457015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.457632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.457913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.458188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.458830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:22.459133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.460760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.461533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 780/1579] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:22.462272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced2d79-14bb-4a99-8909-a9950af4e854 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.471851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced2d79-14bb-4a99-8909-a9950af4e854 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.472778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 800/1580] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:22.473645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.483962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c4f68-d445-4ada-817c-6355f353e890 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.484409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.485020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.485300 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.485572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.486200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.488184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.488786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.489370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.489652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.489975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.490550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.492150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.506881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 801/1581] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:22.506881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.515899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c1a61-f970-4b6d-99a9-0d4ca6660980 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.523434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 781/1582] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.523603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.523718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.523848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.523848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.523848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.523848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.524745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.546970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 802/1583] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.547861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.553784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:22.558866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe241a-acbd-47e2-ae03-ffdf05d21298 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.589351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:22.593448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83188350-2c92-453d-b4bf-c0d7b6eaf856 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.598249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.598865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.599147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.599418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.600085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.602071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.617833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.618560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.618845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.619171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.620338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.622246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.624217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 782/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.625031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.646927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 803/1585] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.646927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.680572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7fe33-3020-4195-b43c-ef9cb896975f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.685020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.685762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.686067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.686340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.700186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5fd75-538f-48ac-889c-0f6430f1c94f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.703847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.704449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.704727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.705005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.713887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.715341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.724708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 804/1586] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:22.725471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.734892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e9d49-aeb7-4c47-9e9a-a123e1e3d8a2 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.739810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.740550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.740838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.741181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.742303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.744952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.745097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.746673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.757698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 783/1587] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:22.758289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.771770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670343c-0aa3-4e7e-bcdd-29de8d552165 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.776800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.777470 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.777754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.778112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.778770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.780597 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 805/1588] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.781622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.783943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.796789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 784/1589] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:22.800616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.812713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33ed5-17af-493c-bbb7-8d2b3e503c74 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.876784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2e45e-9122-44f1-81ab-d07638d335b9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.883174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.884116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.884739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.884859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.885601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.887870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.900056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.901426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.901711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.902112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.903105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.905165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.907931 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:22.922070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 785/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:22.923125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 806/1591] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:22.923254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.924707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.934379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854eac2-2d00-4684-bea8-2714718e044e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.936657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fa695-15aa-40ac-8f87-88f9f03990c2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.940330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.941112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.941420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.941703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.941922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.942541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.942761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.943192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.943956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.944520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.944743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.946794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.957786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 786/1592] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:22.958846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.963553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 807/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:22.964428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:22.979945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538001b7-41b1-485a-af84-ae37f3e3201b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.984743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc917d70-8f3a-4c08-8d95-a02d62668741 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:22.985820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.986683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.986961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.987328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:22.988115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:22.990237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:22.990772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:22.990924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:22.991210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:22.991541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] PATH_INFO: `/v3/projects/91ee5ad0b4064b9d8a038980786072cb/users/703464e4a9524efaa8fdfc1870158cbb/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.011615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ee5ad0b4064b9d8a038980786072cb, user_id=703464e4a9524efaa8fdfc1870158cbb, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.014528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.020838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 787/1594] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:22 2022] POST /identity/v3/users => generated 279 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:23.021730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.028907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:23.036661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ffac3-d161-4a32-9b45-171caa92cfc8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.038092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 808/1595] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:22 2022] PUT /identity/v3/projects/91ee5ad0b4064b9d8a038980786072cb/users/703464e4a9524efaa8fdfc1870158cbb/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:23.038909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.111714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcd785-f818-4946-b81c-c50cfb47bd05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.118233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.120120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.120120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.120467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.121273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.122979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.123723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.124372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.124669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.125000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.126051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.128629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.135167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 809/1596] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:23.136250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.143836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 788/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:23.144838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.151649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf0f1a-0529-495c-863d-d17757846060 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.156965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.157731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.158086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.158302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b63f6-fe5c-45d9-bce4-8ddbb2802f65 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.158562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.159310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.161143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.163380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29313ff-0add-48a6-be82-41a12ce76d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.164016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29313ff-0add-48a6-be82-41a12ce76d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.164293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29313ff-0add-48a6-be82-41a12ce76d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.164562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29313ff-0add-48a6-be82-41a12ce76d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.165243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29313ff-0add-48a6-be82-41a12ce76d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.167461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29313ff-0add-48a6-be82-41a12ce76d06 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.173700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 810/1598] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:23.174540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.186698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 789/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:23.187992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc18f5-8c08-41a0-96a1-4ef45e1d2950 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.188278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc18f5-8c08-41a0-96a1-4ef45e1d2950 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.188553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc18f5-8c08-41a0-96a1-4ef45e1d2950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.194485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1a84f-a554-4545-99ff-7ac6646c1338 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.200104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.200873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.201213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.201492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.202188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.203884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.215912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 811/1600] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:23.216865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3dc18f5-8c08-41a0-96a1-4ef45e1d2950 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:23.218089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3dc18f5-8c08-41a0-96a1-4ef45e1d2950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:23.219622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.230053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf49be-f5cf-4492-8b1c-4e231cfadaec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.234602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 790/1601] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:23.235500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc18f5-8c08-41a0-96a1-4ef45e1d2950 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.241831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.244507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.244812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.245149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.245978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.247973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc18f5-8c08-41a0-96a1-4ef45e1d2950 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.248106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.253270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.254857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.255145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.255414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] PATH_INFO: `/v3/projects/8069217719b746218ee51f6ef07ae1bb/users/fd0156859cc3405b9eca553caa5785e0/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.282410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 812/1602] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:23.283522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.290581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8069217719b746218ee51f6ef07ae1bb, user_id=fd0156859cc3405b9eca553caa5785e0, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.293896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.307361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d449a-14ce-48b6-8313-7d705696e5a2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.317040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 791/1603] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:23 2022] PUT /identity/v3/projects/8069217719b746218ee51f6ef07ae1bb/users/fd0156859cc3405b9eca553caa5785e0/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:23.319900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.426935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373a38e-21cc-4a0f-be11-384fa751413b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.432741 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.433772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.434254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.434734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] PATH_INFO: `/v3/projects/157fb4225c2546c2895715756c36d70c/users/a7cbf89bdd8c4e0e9be7ac5f8567538d/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.438718 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.439868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.439868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.440079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] PATH_INFO: `/v3/projects/1fa73b0bfee14a9ea25f03e523078995/users/f8f44df824414e5ebc8cfd2126e9d66f/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.463985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=157fb4225c2546c2895715756c36d70c, user_id=a7cbf89bdd8c4e0e9be7ac5f8567538d, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.467155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.475585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa73b0bfee14a9ea25f03e523078995, user_id=f8f44df824414e5ebc8cfd2126e9d66f, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.478601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.492474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 792/1604] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:23 2022] PUT /identity/v3/projects/157fb4225c2546c2895715756c36d70c/users/a7cbf89bdd8c4e0e9be7ac5f8567538d/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:23.496817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.516264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 813/1605] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:23 2022] PUT /identity/v3/projects/1fa73b0bfee14a9ea25f03e523078995/users/f8f44df824414e5ebc8cfd2126e9d66f/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:23.518914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.520449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:23.604568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e664-6749-4511-a33c-5736de7f3e7b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.605085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:23.621469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd4f66-2573-46cf-9e21-75748d376458 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.627979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6038c-b9c0-4ace-890d-2c6ecb0a6297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.629303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6038c-b9c0-4ace-890d-2c6ecb0a6297 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.629829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6038c-b9c0-4ace-890d-2c6ecb0a6297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.630353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6038c-b9c0-4ace-890d-2c6ecb0a6297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.686749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.687387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.687675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.687942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.688613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.690489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.705460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 793/1606] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:23.708967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.718638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a254bf-6230-451c-967e-2f0084ceada2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:23.721205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6038c-b9c0-4ace-890d-2c6ecb0a6297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.723525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4b8f2-7405-4d62-8792-fdf3df96561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:23.724242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4b8f2-7405-4d62-8792-fdf3df96561f admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.724706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4b8f2-7405-4d62-8792-fdf3df96561f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.724873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4b8f2-7405-4d62-8792-fdf3df96561f admin admin] PATH_INFO: `/v3/projects/2002558f3d814b1b892ca2d22ff25f05/users/0354d80eb898418fae8acf8baf4894b1/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.739611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6038c-b9c0-4ace-890d-2c6ecb0a6297 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.741003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4b8f2-7405-4d62-8792-fdf3df96561f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2002558f3d814b1b892ca2d22ff25f05, user_id=0354d80eb898418fae8acf8baf4894b1, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:23.743776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4b8f2-7405-4d62-8792-fdf3df96561f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:23.754503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 814/1607] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:23.756855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d4dfa-ce5f-4776-a36b-494f8e585712 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.757397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d4dfa-ce5f-4776-a36b-494f8e585712 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.757907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d4dfa-ce5f-4776-a36b-494f8e585712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.762022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 794/1608] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:23 2022] PUT /identity/v3/projects/2002558f3d814b1b892ca2d22ff25f05/users/0354d80eb898418fae8acf8baf4894b1/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:23.763174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4c52d-32af-49ad-93f2-cca322f1cd30 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.763541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4c52d-32af-49ad-93f2-cca322f1cd30 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.763842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4c52d-32af-49ad-93f2-cca322f1cd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.800974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-92d4c52d-32af-49ad-93f2-cca322f1cd30 None None] MFA Rules not processed for user `a7cbf89bdd8c4e0e9be7ac5f8567538d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:23.802297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3e1d4dfa-ce5f-4776-a36b-494f8e585712 None None] MFA Rules not processed for user `fd0156859cc3405b9eca553caa5785e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:23.802955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-92d4c52d-32af-49ad-93f2-cca322f1cd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:23.803677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e1d4dfa-ce5f-4776-a36b-494f8e585712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:23.893167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 795/1609] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:23.894814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d677c-39e7-4664-8434-97815a6956bc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:23.895437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d677c-39e7-4664-8434-97815a6956bc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:23.895437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d677c-39e7-4664-8434-97815a6956bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:23.899284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 815/1610] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:23.900141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d4dfa-ce5f-4776-a36b-494f8e585712 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:23.957874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-615d677c-39e7-4664-8434-97815a6956bc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:23.962313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-615d677c-39e7-4664-8434-97815a6956bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:23.978187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:24.007892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d4dfa-ce5f-4776-a36b-494f8e585712 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.014334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.015047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.015331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.015856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] PATH_INFO: `/v3/projects/251dafdc7ff74ee98d6d76c540d28e49/users/ba1ba449c2d8465ba93c8091ae5eb028/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.040747 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=251dafdc7ff74ee98d6d76c540d28e49, user_id=ba1ba449c2d8465ba93c8091ae5eb028, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.045359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:24.057830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 796/1611] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:24.057830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c666cf-8a88-484a-870d-db9e40caccbf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.060259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c666cf-8a88-484a-870d-db9e40caccbf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.060259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c666cf-8a88-484a-870d-db9e40caccbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.075787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 816/1612] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:23 2022] PUT /identity/v3/projects/251dafdc7ff74ee98d6d76c540d28e49/users/ba1ba449c2d8465ba93c8091ae5eb028/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:24.075787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.081622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b1c666cf-8a88-484a-870d-db9e40caccbf None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:24.082937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b1c666cf-8a88-484a-870d-db9e40caccbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:24.089679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:24.162420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55fa05-35ee-48d2-8ad7-30e5fd8be41d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.169793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a28788-6db8-4ccf-8ebf-b53718f2a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.170910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a28788-6db8-4ccf-8ebf-b53718f2a8ba admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.171421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a28788-6db8-4ccf-8ebf-b53718f2a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.171918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a28788-6db8-4ccf-8ebf-b53718f2a8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.172873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a28788-6db8-4ccf-8ebf-b53718f2a8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.175335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a28788-6db8-4ccf-8ebf-b53718f2a8ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:24.181612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 797/1613] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:24.181612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5bdb4-ab50-41c0-a7c7-c571fe79f683 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.181612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5bdb4-ab50-41c0-a7c7-c571fe79f683 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.181612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5bdb4-ab50-41c0-a7c7-c571fe79f683 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.183392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 798/1614] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 6 01:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:24.184612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5bdb4-ab50-41c0-a7c7-c571fe79f683 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.225579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 817/1615] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:24 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:24.226826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd4a1-e262-46bd-b79d-736acbd4cb7c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.227110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd4a1-e262-46bd-b79d-736acbd4cb7c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.227744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd4a1-e262-46bd-b79d-736acbd4cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.273869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8b2fd4a1-e262-46bd-b79d-736acbd4cb7c None None] MFA Rules not processed for user `ba1ba449c2d8465ba93c8091ae5eb028`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:24.276827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b2fd4a1-e262-46bd-b79d-736acbd4cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:24.295873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5bdb4-ab50-41c0-a7c7-c571fe79f683 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.385918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f13553-690c-4d2a-ade1-a69c1f854cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.386916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13553-690c-4d2a-ade1-a69c1f854cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.387345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 818/1616] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:24.388535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd4a1-e262-46bd-b79d-736acbd4cb7c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.391038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13553-690c-4d2a-ade1-a69c1f854cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.391430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13553-690c-4d2a-ade1-a69c1f854cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.403011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd4a1-e262-46bd-b79d-736acbd4cb7c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.410803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.411592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.411906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.412186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.432348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:24.492413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f13553-690c-4d2a-ade1-a69c1f854cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.494185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f13553-690c-4d2a-ade1-a69c1f854cc0 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:24.516578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 799/1617] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:24.519823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e4262-fd51-4dae-946b-7d1ffed8ba1b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.520115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e4262-fd51-4dae-946b-7d1ffed8ba1b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.520469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e4262-fd51-4dae-946b-7d1ffed8ba1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.523192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.524969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:24.551668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 819/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:24.552558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.590254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7c5e4262-fd51-4dae-946b-7d1ffed8ba1b None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:24.591628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09fb70-b3b5-4546-8255-3e647f0350c0 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.592761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7c5e4262-fd51-4dae-946b-7d1ffed8ba1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:24.597938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.598774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.599120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.599387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.600077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.602185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:24.627978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:24.638545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 820/1619] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:24 2022] POST /identity/v3/users => generated 292 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:24.638545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.651481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063cfec-d42f-4741-a3fc-cff4a9d03a8d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.657341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.657958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.658232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.658502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.659138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.661465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:24.738613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 821/1620] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:24 2022] POST /identity/v3/projects => generated 357 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:24.740228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.767303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 800/1621] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 6 01:11:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:24.768616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e4262-fd51-4dae-946b-7d1ffed8ba1b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.784814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e4262-fd51-4dae-946b-7d1ffed8ba1b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.845175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282e8e-48e6-4659-81dc-0e043141af7b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:24.862568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.863412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.863704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.864096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.864767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.866809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:24.930851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 822/1622] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:24.930851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:24.951530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:24.953772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:24.954263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:24.954684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:24.955733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:24.958580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.028913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 801/1623] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:24 2022] POST /identity/v3/users => generated 285 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:25.029841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:25.062018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf11c0-31cd-44b6-874b-bce4b07f2bcb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:25.068616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:25.076331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.076615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.076892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] PATH_INFO: `/v3/projects/936f93fa6e5d4837b159c8f188b947c6/users/eefc22f28680489f8780dd5b557ad846/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.134530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=936f93fa6e5d4837b159c8f188b947c6, user_id=eefc22f28680489f8780dd5b557ad846, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:25.145247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.187501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 802/1624] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:25 2022] PUT /identity/v3/projects/936f93fa6e5d4837b159c8f188b947c6/users/eefc22f28680489f8780dd5b557ad846/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:25.188889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:25.204519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d084c3-77a6-4ebc-8885-ff825e6cddd9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:25.242572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:25.343202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c5986-b796-4b9e-8f67-82a62ee8c2eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:25.343638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c5986-b796-4b9e-8f67-82a62ee8c2eb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.343960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c5986-b796-4b9e-8f67-82a62ee8c2eb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.344311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c5986-b796-4b9e-8f67-82a62ee8c2eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.352947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:25.410907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b45b-67c9-48b4-a588-a52ecca181f3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:25.420964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:25.421864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.422982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.422982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.423553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:25.425609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.436343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c5986-b796-4b9e-8f67-82a62ee8c2eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:25.439238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c5986-b796-4b9e-8f67-82a62ee8c2eb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.453502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 803/1625] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:25.453502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:25.457530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 823/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:25.459213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fbf3d-6c64-482f-80b8-62f99c20a412 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.459672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fbf3d-6c64-482f-80b8-62f99c20a412 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.460106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fbf3d-6c64-482f-80b8-62f99c20a412 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.466969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 824/1627] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 6 01:11:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:25.468886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fbf3d-6c64-482f-80b8-62f99c20a412 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:25.475166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd1c1d-b873-44e7-aef5-f4d1269f2eb0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:25.490150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fbf3d-6c64-482f-80b8-62f99c20a412 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:25.490307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee7d86-d488-4abe-8329-68b1520cb28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:25.491140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee7d86-d488-4abe-8329-68b1520cb28c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.491692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee7d86-d488-4abe-8329-68b1520cb28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.492386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee7d86-d488-4abe-8329-68b1520cb28c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.493109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee7d86-d488-4abe-8329-68b1520cb28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:25.497537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:25.497963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee7d86-d488-4abe-8329-68b1520cb28c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.498236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.498520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.498795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] PATH_INFO: `/v3/projects/bb90bff777894d16a793ad4bcf508074/users/08a9eb114eb14b35a22e3ea4160552c6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.523809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 804/1628] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:25.523809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fb490-ea11-4fa2-b887-3622bc7302a2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.524116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fb490-ea11-4fa2-b887-3622bc7302a2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.524364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fb490-ea11-4fa2-b887-3622bc7302a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.536982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb90bff777894d16a793ad4bcf508074, user_id=08a9eb114eb14b35a22e3ea4160552c6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:25.539681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.568886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-791fb490-ea11-4fa2-b887-3622bc7302a2 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:25.571155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 825/1629] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:25 2022] PUT /identity/v3/projects/bb90bff777894d16a793ad4bcf508074/users/08a9eb114eb14b35a22e3ea4160552c6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:25.572363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:25.577700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-791fb490-ea11-4fa2-b887-3622bc7302a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:25.701860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93b769-45b0-4b86-b9a3-0d7bab4c0a4d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:25.710643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e92d6-337d-41e1-b3c9-ed9898051e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:25.711553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e92d6-337d-41e1-b3c9-ed9898051e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.712045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e92d6-337d-41e1-b3c9-ed9898051e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.712521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e92d6-337d-41e1-b3c9-ed9898051e77 admin admin] PATH_INFO: `/v3/projects/936f93fa6e5d4837b159c8f188b947c6/users/eefc22f28680489f8780dd5b557ad846/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.714668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e92d6-337d-41e1-b3c9-ed9898051e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=936f93fa6e5d4837b159c8f188b947c6, user_id=eefc22f28680489f8780dd5b557ad846, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:25.717530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e92d6-337d-41e1-b3c9-ed9898051e77 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.731743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 805/1630] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 6 01:11:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:25.741210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fb490-ea11-4fa2-b887-3622bc7302a2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:25.742126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 826/1631] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:25 2022] PUT /identity/v3/projects/936f93fa6e5d4837b159c8f188b947c6/users/eefc22f28680489f8780dd5b557ad846/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:25.745221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb4524-54dd-4bc9-9ede-877f33b3651f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.746496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb4524-54dd-4bc9-9ede-877f33b3651f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.747630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb4524-54dd-4bc9-9ede-877f33b3651f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.749605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 827/1632] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 6 01:11:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:25.750890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6548250-a790-4d1f-951b-dc9c84ae1711 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.751346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6548250-a790-4d1f-951b-dc9c84ae1711 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.752290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6548250-a790-4d1f-951b-dc9c84ae1711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.754597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fb490-ea11-4fa2-b887-3622bc7302a2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:25.789709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b6548250-a790-4d1f-951b-dc9c84ae1711 None None] MFA Rules not processed for user `eefc22f28680489f8780dd5b557ad846`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:25.792120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b6548250-a790-4d1f-951b-dc9c84ae1711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:25.892168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:25.893091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.894222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.894671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.895506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:25.899462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 828/1633] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:25.901421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa951474-c015-44d0-a021-84e4af940e23 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:25.901730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa951474-c015-44d0-a021-84e4af940e23 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:25.902136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa951474-c015-44d0-a021-84e4af940e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:25.912026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:25.912633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:25.931084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 806/1634] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:25.933885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:25.946048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fa951474-c015-44d0-a021-84e4af940e23 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:25.947487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fa951474-c015-44d0-a021-84e4af940e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:25.987092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:26.070823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83944d-c196-45b3-acd3-5124dd67b9ad admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:26.085090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:26.085947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:26.086544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:26.086825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:26.111063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:26.112558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:26.144917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 807/1635] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:26.147172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:26.192269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 829/1636] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:11:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:26.195928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6006b-3dde-4554-9761-d5d54e9dcfcd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:26.196846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6006b-3dde-4554-9761-d5d54e9dcfcd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:26.200455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6006b-3dde-4554-9761-d5d54e9dcfcd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:26.200939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc19f7-64a2-4bf8-921a-bfabc3d8d396 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:26.212696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 830/1637] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 6 01:11:26 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:26.215807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6006b-3dde-4554-9761-d5d54e9dcfcd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:26.223833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d1d6d-1bd2-4b0b-9e6a-e5c7893569ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:26.225634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d1d6d-1bd2-4b0b-9e6a-e5c7893569ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:26.226434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d1d6d-1bd2-4b0b-9e6a-e5c7893569ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:26.226729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d1d6d-1bd2-4b0b-9e6a-e5c7893569ab admin admin] PATH_INFO: `/v3/projects/bb90bff777894d16a793ad4bcf508074/users/08a9eb114eb14b35a22e3ea4160552c6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:26.232674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d1d6d-1bd2-4b0b-9e6a-e5c7893569ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb90bff777894d16a793ad4bcf508074, user_id=08a9eb114eb14b35a22e3ea4160552c6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:26.241359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d1d6d-1bd2-4b0b-9e6a-e5c7893569ab admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:26.246672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6006b-3dde-4554-9761-d5d54e9dcfcd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:26.253896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f828c4-d37a-4d58-9a18-3aeb8497694c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:26.254803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f828c4-d37a-4d58-9a18-3aeb8497694c service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:26.254803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f828c4-d37a-4d58-9a18-3aeb8497694c service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:26.255082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f828c4-d37a-4d58-9a18-3aeb8497694c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:26.282916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 808/1638] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:26 2022] PUT /identity/v3/projects/bb90bff777894d16a793ad4bcf508074/users/08a9eb114eb14b35a22e3ea4160552c6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:26.293404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1313934-a340-4b85-a419-958880ea7ea1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:26.295165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1313934-a340-4b85-a419-958880ea7ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:26.295435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1313934-a340-4b85-a419-958880ea7ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:26.304109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f828c4-d37a-4d58-9a18-3aeb8497694c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:26.307781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f828c4-d37a-4d58-9a18-3aeb8497694c service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:26.348462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 831/1639] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:26.355487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3730b-ac99-4ee0-81f1-e4946d36caa2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:26.355799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3730b-ac99-4ee0-81f1-e4946d36caa2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:26.356108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3730b-ac99-4ee0-81f1-e4946d36caa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:26.368941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e1313934-a340-4b85-a419-958880ea7ea1 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:26.372242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e1313934-a340-4b85-a419-958880ea7ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:26.472663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-08b3730b-ac99-4ee0-81f1-e4946d36caa2 None None] MFA Rules not processed for user `08a9eb114eb14b35a22e3ea4160552c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:26.473905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-08b3730b-ac99-4ee0-81f1-e4946d36caa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:26.626120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 809/1640] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:11:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:26.660066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1313934-a340-4b85-a419-958880ea7ea1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:26.758484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 832/1641] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:26.807494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1313934-a340-4b85-a419-958880ea7ea1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:26.821866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:26.824583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:26.826868 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:26.827787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:26.843633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:26.845994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:26.868185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 810/1642] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:11:27.137323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3730b-ac99-4ee0-81f1-e4946d36caa2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:27.153328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3730b-ac99-4ee0-81f1-e4946d36caa2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:27.160034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:27.160945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:27.162039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:27.163275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:27.178779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:27.182433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:27.217163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 833/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:27.470745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:27.485490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa44b5-9c82-48fa-9387-c6c632014295 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:27.494797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134e64d-9100-4656-a525-b0549f087585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:27.495421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134e64d-9100-4656-a525-b0549f087585 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:27.495819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134e64d-9100-4656-a525-b0549f087585 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:27.496121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134e64d-9100-4656-a525-b0549f087585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:27.597055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:27.655976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134e64d-9100-4656-a525-b0549f087585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:27.660116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134e64d-9100-4656-a525-b0549f087585 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:27.698008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 811/1644] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:28.175724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:28.280583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:28.327208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5a7e1-7198-4bdf-851e-13ae8b582897 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:28.327208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:28.327208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:28.327208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:28.327741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] PATH_INFO: `/v3/users/fd0156859cc3405b9eca553caa5785e0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:28.327741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0156859cc3405b9eca553caa5785e0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:28.327741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:11:28.327741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 06 01:11:28.327741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: Sep 06 01:11:28.327741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:28.469684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0156859cc3405b9eca553caa5785e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:28.497357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0156859cc3405b9eca553caa5785e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:28.547299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0156859cc3405b9eca553caa5785e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:28.548568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0156859cc3405b9eca553caa5785e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:28.577079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0156859cc3405b9eca553caa5785e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:28.600778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 834/1645] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:28 2022] DELETE /identity/v3/users/fd0156859cc3405b9eca553caa5785e0 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:28.603148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:28.604177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:28.604282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:28.645276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:28.646757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:28.774348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 812/1646] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:28.862562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:28.865132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:29.019713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d430f-e799-4a39-b5f9-6110be720f3b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:29.032685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:29.033498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:29.033890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:29.034268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:29.057234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:29.059096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:29.085689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 835/1647] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:11:31.069641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:31.071279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:31.100345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c3880-b82e-4343-a3d6-7bd0ad28e829 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:31.112459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:31.113138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:31.113406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:31.113683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] PATH_INFO: `/v3/projects/8069217719b746218ee51f6ef07ae1bb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:31.114858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8069217719b746218ee51f6ef07ae1bb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:31.115980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 01:11:31.115980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 06 01:11:31.115980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: Sep 06 01:11:31.117385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:31.317265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8069217719b746218ee51f6ef07ae1bb', 'request_id': 'req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:31.337938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8069217719b746218ee51f6ef07ae1bb', 'request_id': 'req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:31.373818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8069217719b746218ee51f6ef07ae1bb', 'request_id': 'req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:31.432342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] The token cache is being invalidate because project 8069217719b746218ee51f6ef07ae1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:31.432654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:31.435343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:31.444125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:31.444654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 813/1648] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:31 2022] DELETE /identity/v3/projects/8069217719b746218ee51f6ef07ae1bb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:31.461000 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dc52f-5e5b-43c0-a62b-fb706472685f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:31.764777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:31.767330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:31.768392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:31.771107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] PATH_INFO: `/v3/users/eefc22f28680489f8780dd5b557ad846` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:31.775968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=eefc22f28680489f8780dd5b557ad846)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:31.781281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:31.865691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:31.969591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eefc22f28680489f8780dd5b557ad846'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:31.989205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eefc22f28680489f8780dd5b557ad846'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:32.041543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eefc22f28680489f8780dd5b557ad846'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:32.044935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eefc22f28680489f8780dd5b557ad846'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:32.071607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cee9f588-e437-4866-b7f8-338ad6ca32fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eefc22f28680489f8780dd5b557ad846'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:32.086811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 836/1649] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:31 2022] DELETE /identity/v3/users/eefc22f28680489f8780dd5b557ad846 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:11:34.488685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:34.490127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:34.519658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e8d7e-6d38-4125-b9a3-d3834de9ac85 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:34.763926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:34.764588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:34.764877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:34.765147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] PATH_INFO: `/v3/projects/936f93fa6e5d4837b159c8f188b947c6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:34.766946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] RBAC: Authorizing `identity:delete_project(project_id=936f93fa6e5d4837b159c8f188b947c6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:34.769105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:34.798642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:34.897360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936f93fa6e5d4837b159c8f188b947c6', 'request_id': 'req-0cc641a8-e315-4f2f-904e-bd13b9b7732c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:34.916765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936f93fa6e5d4837b159c8f188b947c6', 'request_id': 'req-0cc641a8-e315-4f2f-904e-bd13b9b7732c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:34.934680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936f93fa6e5d4837b159c8f188b947c6', 'request_id': 'req-0cc641a8-e315-4f2f-904e-bd13b9b7732c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:34.960140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] The token cache is being invalidate because project 936f93fa6e5d4837b159c8f188b947c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:34.960435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:34.961400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0cc641a8-e315-4f2f-904e-bd13b9b7732c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:34.964847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 814/1650] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:34 2022] DELETE /identity/v3/projects/936f93fa6e5d4837b159c8f188b947c6 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:34.969918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630420ab-5169-4299-a90e-ad104821d16c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:34.970752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630420ab-5169-4299-a90e-ad104821d16c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:34.971191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630420ab-5169-4299-a90e-ad104821d16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.046897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-630420ab-5169-4299-a90e-ad104821d16c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:35.051793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-630420ab-5169-4299-a90e-ad104821d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:35.217066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 837/1651] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:35.226093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751bc9-48ba-477e-ae15-f5a311244279 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.226525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751bc9-48ba-477e-ae15-f5a311244279 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.226816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751bc9-48ba-477e-ae15-f5a311244279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.284914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-27751bc9-48ba-477e-ae15-f5a311244279 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:35.286164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-27751bc9-48ba-477e-ae15-f5a311244279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:35.321405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 815/1652] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:35.325893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630420ab-5169-4299-a90e-ad104821d16c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:35.356837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630420ab-5169-4299-a90e-ad104821d16c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:35.373586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:35.377005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.377524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.378160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.382034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:35.383099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:35.388325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:35.424050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 838/1653] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:35.428416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27751bc9-48ba-477e-ae15-f5a311244279 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:35.443513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27751bc9-48ba-477e-ae15-f5a311244279 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:35.460501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:35.463170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.464168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.464489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.468598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:35.475591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:35.599097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 816/1654] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:35 2022] POST /identity/v3/projects => generated 363 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:35.601747 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:35.622714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f4e04-2ff2-49ce-a7eb-cfeb22238abb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:35.804921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96e848-aa25-4847-9488-20faa9635c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:35.806001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e848-aa25-4847-9488-20faa9635c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.806322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e848-aa25-4847-9488-20faa9635c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.806913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e848-aa25-4847-9488-20faa9635c87 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.807742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96e848-aa25-4847-9488-20faa9635c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:35.811204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96e848-aa25-4847-9488-20faa9635c87 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:35.854734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 839/1655] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:35 2022] POST /identity/v3/users => generated 282 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:35.856501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:35.858435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.858708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.858989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.869914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca2115-118d-4e03-93e0-b7982ca349c2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:35.876965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:35.877856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.878656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.879131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.880212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:35.882051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:35.898451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 817/1656] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:35.904379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:35.905250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:35.931417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ade75-bb2b-4282-a5f8-029d5edd1f4a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:35.938248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:35.939168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.940296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.940743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:35.943178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:35.945363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:35.947492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] MFA Rules not processed for user `f8f44df824414e5ebc8cfd2126e9d66f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:35.953107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:35.967070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 818/1657] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:35.969646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:35.981440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81cf1f-fc88-470c-b03c-c1e3af602859 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:35.986983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77ffd9-1fdf-4665-9d7a-95ec249bf6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:35.987769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77ffd9-1fdf-4665-9d7a-95ec249bf6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:35.988128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77ffd9-1fdf-4665-9d7a-95ec249bf6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:35.988493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77ffd9-1fdf-4665-9d7a-95ec249bf6bb admin admin] PATH_INFO: `/v3/projects/c24534055771473ebdf921acdbaf4b73/users/928e347b25664d7891e45aa6066ba58f/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.010957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77ffd9-1fdf-4665-9d7a-95ec249bf6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24534055771473ebdf921acdbaf4b73, user_id=928e347b25664d7891e45aa6066ba58f, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:36.014268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77ffd9-1fdf-4665-9d7a-95ec249bf6bb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:36.040925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 819/1658] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:35 2022] PUT /identity/v3/projects/c24534055771473ebdf921acdbaf4b73/users/928e347b25664d7891e45aa6066ba58f/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:36.050996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:36.051286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:36.051566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.113144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] MFA Rules not processed for user `928e347b25664d7891e45aa6066ba58f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:36.114379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:36.148704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 840/1659] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:36.163472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:36.171896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:36.286730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 820/1660] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:36.287874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:36.290924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:36.315852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:36.350724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8372c2b-d5fd-47c3-8f86-a5a0f1ef9bbc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:36.409191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b0aed-1c65-41dc-9658-bde846ad6db9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:36.414545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:36.415338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:36.416431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:36.416814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] PATH_INFO: `/v3/users/a7cbf89bdd8c4e0e9be7ac5f8567538d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.418173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7cbf89bdd8c4e0e9be7ac5f8567538d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:36.420767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:36.470562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fb097-0633-4c50-886c-c3c3ab6e1e7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:36.471533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fb097-0633-4c50-886c-c3c3ab6e1e7a service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:36.471986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fb097-0633-4c50-886c-c3c3ab6e1e7a service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:36.472394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fb097-0633-4c50-886c-c3c3ab6e1e7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.496039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7cbf89bdd8c4e0e9be7ac5f8567538d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:36.497105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7cbf89bdd8c4e0e9be7ac5f8567538d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:36.510591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7cbf89bdd8c4e0e9be7ac5f8567538d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:36.547481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7cbf89bdd8c4e0e9be7ac5f8567538d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:36.548524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7cbf89bdd8c4e0e9be7ac5f8567538d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:36.550198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:36.570871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 821/1661] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:36 2022] DELETE /identity/v3/users/a7cbf89bdd8c4e0e9be7ac5f8567538d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:36.572850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:36.576285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:36.616564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fb097-0633-4c50-886c-c3c3ab6e1e7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:36.620292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fb097-0633-4c50-886c-c3c3ab6e1e7a service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:36.651127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 841/1662] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:36.656212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:36.656495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:36.656761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.690750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453673a1-098d-4032-84f1-2e9695ab990e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:36.696391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:36.697855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:36.698258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:36.698737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.699724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:36.702108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:36.711614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:36.714913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:36.744335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 842/1663] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:36.750003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:36.753210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:36.769778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 822/1664] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:36 2022] POST /identity/v3/projects => generated 363 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:36.770738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:36.773008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:36.795946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cf4d4-224d-4d7c-a5f9-f9b448dc929d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:36.814235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:36.936800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3e4e-bef7-4757-8c81-c34ea7d26ef7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:36.945857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:36.946544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:36.948044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:36.948331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.960077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:36.963078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:36.963506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:36.964330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] PATH_INFO: `/v3/users/540fc07c6373415fb314c8af5010a936` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:36.968250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=540fc07c6373415fb314c8af5010a936)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:36.975080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:36.975268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:36.979603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:37.038206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 823/1665] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:37.039062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:37.091979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff53d36-f6c6-484c-9719-2b4c11260603 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:37.124824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a0390-7122-44d3-a52b-cb17e7c634a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:37.125461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a0390-7122-44d3-a52b-cb17e7c634a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:37.125743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a0390-7122-44d3-a52b-cb17e7c634a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:37.126019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a0390-7122-44d3-a52b-cb17e7c634a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:37.126662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a0390-7122-44d3-a52b-cb17e7c634a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:37.128658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a0390-7122-44d3-a52b-cb17e7c634a7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:37.169674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 824/1666] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:37 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:37.171084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f956c-3c9a-4091-b6f3-9bce9087443f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:37.171374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f956c-3c9a-4091-b6f3-9bce9087443f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:37.171652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f956c-3c9a-4091-b6f3-9bce9087443f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:37.176099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '540fc07c6373415fb314c8af5010a936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:37.200419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '540fc07c6373415fb314c8af5010a936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:37.230707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '540fc07c6373415fb314c8af5010a936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:37.231480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '540fc07c6373415fb314c8af5010a936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:37.253055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '540fc07c6373415fb314c8af5010a936'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:37.274245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 843/1667] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:36 2022] DELETE /identity/v3/users/540fc07c6373415fb314c8af5010a936 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:37.277986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:37.282908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:37.283466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b80f956c-3c9a-4091-b6f3-9bce9087443f None None] MFA Rules not processed for user `2dc5c46544a447159b7fd3d225cd9697`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:37.284985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b80f956c-3c9a-4091-b6f3-9bce9087443f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:37.450915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:37.477860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 825/1668] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:37.479642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7696a77-bbce-4830-bd8f-03358831cf4c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:37.480184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7696a77-bbce-4830-bd8f-03358831cf4c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:37.480656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7696a77-bbce-4830-bd8f-03358831cf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:37.493753 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c61fb-fbb0-4c1f-a7b7-f6c3cb3c7736 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:37.511430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:37.512015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:37.512299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:37.512717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:37.513534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:37.515357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:37.532937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a7696a77-bbce-4830-bd8f-03358831cf4c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:37.534498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a7696a77-bbce-4830-bd8f-03358831cf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:37.541397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 844/1669] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:37.551797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:37.553455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 826/1670] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:37.554951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7696a77-bbce-4830-bd8f-03358831cf4c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:37.557852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:37.591385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7696a77-bbce-4830-bd8f-03358831cf4c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:37.607056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:37.608155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:37.609865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:37.610359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] PATH_INFO: `/v3/projects/f9f86fe3a6b4468da862006aa4cbe4f1/users/1a39d55215274e7ea6d5431654e5f029/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:37.670238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f86fe3a6b4468da862006aa4cbe4f1, user_id=1a39d55215274e7ea6d5431654e5f029, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:37.679178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:37.706391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 827/1671] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:37 2022] PUT /identity/v3/projects/f9f86fe3a6b4468da862006aa4cbe4f1/users/1a39d55215274e7ea6d5431654e5f029/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:37.707572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:37.714156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:37.762118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134d0f3-4937-4ebf-b557-96ff9a9d5905 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:37.856368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bae99-1fb3-4330-a085-e0e4e5d51d2c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:37.865727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:37.866366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:37.866751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:37.867129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:37.898312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:37.909677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:37.910480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:37.910767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:37.911042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.011992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.014091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.054665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 828/1672] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:38.065817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:38.106913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.119366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d421554-9ad2-47b6-879c-6b1662682891 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:38.123204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.135315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:38.136410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.136806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.137376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.138286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.140502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.157055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 845/1673] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:38.158131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:38.159926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:38.173758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 829/1674] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:38.182823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:38.190719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a82e7-0391-4ba6-99dd-640999eae89f service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:38.211003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:38.213231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.213731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec244eb3-b746-4208-afb5-1bdd1940da1e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:38.215085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.215566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] PATH_INFO: `/v3/projects/157fb4225c2546c2895715756c36d70c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.218145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=157fb4225c2546c2895715756c36d70c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.221523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d09355-1a40-4966-b0c8-bfafc17e1fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:38.222389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.222808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d09355-1a40-4966-b0c8-bfafc17e1fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.223610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d09355-1a40-4966-b0c8-bfafc17e1fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.224148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d09355-1a40-4966-b0c8-bfafc17e1fe8 admin admin] PATH_INFO: `/v3/projects/f9f86fe3a6b4468da862006aa4cbe4f1/users/1a39d55215274e7ea6d5431654e5f029/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.226583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d09355-1a40-4966-b0c8-bfafc17e1fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f86fe3a6b4468da862006aa4cbe4f1, user_id=1a39d55215274e7ea6d5431654e5f029, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.231996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d09355-1a40-4966-b0c8-bfafc17e1fe8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.257678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:38.286303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 830/1675] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:38 2022] PUT /identity/v3/projects/f9f86fe3a6b4468da862006aa4cbe4f1/users/1a39d55215274e7ea6d5431654e5f029/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:38.289187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.289484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.289763 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.338392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157fb4225c2546c2895715756c36d70c', 'request_id': 'req-b283d903-5013-42b9-a3b9-3dba924a4ac6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:38.351095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157fb4225c2546c2895715756c36d70c', 'request_id': 'req-b283d903-5013-42b9-a3b9-3dba924a4ac6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:38.363488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157fb4225c2546c2895715756c36d70c', 'request_id': 'req-b283d903-5013-42b9-a3b9-3dba924a4ac6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:38.395261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] The token cache is being invalidate because project 157fb4225c2546c2895715756c36d70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:38.395548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:38.396580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b283d903-5013-42b9-a3b9-3dba924a4ac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:38.398196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 846/1676] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:38 2022] DELETE /identity/v3/projects/157fb4225c2546c2895715756c36d70c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:38.403866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c312846-9f9e-4dae-93c8-5134c7b492e9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.404281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c312846-9f9e-4dae-93c8-5134c7b492e9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.404658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c312846-9f9e-4dae-93c8-5134c7b492e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.411689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] MFA Rules not processed for user `1a39d55215274e7ea6d5431654e5f029`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:38.422442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:38.453412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8c312846-9f9e-4dae-93c8-5134c7b492e9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:38.455217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c312846-9f9e-4dae-93c8-5134c7b492e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:38.566883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 847/1677] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:38.576249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90666dea-0d10-46a0-9f54-95c039e783ca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.576688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90666dea-0d10-46a0-9f54-95c039e783ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.577077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90666dea-0d10-46a0-9f54-95c039e783ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.608728 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-90666dea-0d10-46a0-9f54-95c039e783ca None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:38.610104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90666dea-0d10-46a0-9f54-95c039e783ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:38.621591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 848/1678] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:38.625729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90666dea-0d10-46a0-9f54-95c039e783ca None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:38.636087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90666dea-0d10-46a0-9f54-95c039e783ca None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:38.641902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:38.642657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.643026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.643370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.644150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:38.644538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.646453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.668989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 849/1679] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:11:38.670369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:38.691770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c733288-371a-4275-a54b-0212a0615ef7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:38.700620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:38.702351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.702859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.703257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.704875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.708607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.766109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 850/1680] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:38 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:38.767341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:38.803057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 831/1681] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:11:38.814406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:38.816088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:38.966151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13419964-ac07-44d4-ba30-d593df63493b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:38.972506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:38.973140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:38.973427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:38.973936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:38.974859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:38.977274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:38.990284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:38.997544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad50a11-297e-4c1e-bd48-4bd248016d4b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:39.021459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 851/1682] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:38 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:39.023575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:39.057966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74522-8848-4a5a-a776-28aa0cb4beea admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:39.061514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:39.073910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:39.076595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:39.076888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:39.079477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:39.081015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:39.085878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:39.123218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 852/1683] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:39.124894 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:39.137467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5e7b5-8da3-4997-a473-302bdebdbb62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:39.138768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5e7b5-8da3-4997-a473-302bdebdbb62 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:39.140254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5e7b5-8da3-4997-a473-302bdebdbb62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:39.141501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5e7b5-8da3-4997-a473-302bdebdbb62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:39.152005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40300be-d942-4382-85f4-f8e459e058fc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:39.166952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:39.168269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:39.168798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:39.171267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:39.173363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:39.176118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:39.208069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 853/1684] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:39.209074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:39.237317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238ba3-0815-4e11-979b-dc4edb74425f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:39.251127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:39.253216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:39.254891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:39.255206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] PATH_INFO: `/v3/projects/80d68779ebb94b8cbe0f142391e62216/users/660e79e294b241dd8087a8cfa2725f98/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:39.298371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5e7b5-8da3-4997-a473-302bdebdbb62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:39.300136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5e7b5-8da3-4997-a473-302bdebdbb62 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:39.301246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d68779ebb94b8cbe0f142391e62216, user_id=660e79e294b241dd8087a8cfa2725f98, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:39.305741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:39.330211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 832/1685] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:11:39.352255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 854/1686] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:39 2022] PUT /identity/v3/projects/80d68779ebb94b8cbe0f142391e62216/users/660e79e294b241dd8087a8cfa2725f98/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:39.353313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc785061-aeac-4108-82c8-b3df0f344ca9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:39.353652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc785061-aeac-4108-82c8-b3df0f344ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:39.353930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc785061-aeac-4108-82c8-b3df0f344ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:39.416995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cc785061-aeac-4108-82c8-b3df0f344ca9 None None] MFA Rules not processed for user `660e79e294b241dd8087a8cfa2725f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:39.418257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc785061-aeac-4108-82c8-b3df0f344ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:39.470899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:39.635560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 833/1687] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:39.642184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc785061-aeac-4108-82c8-b3df0f344ca9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:39.735656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c29d3-80b3-4e72-87cc-f4b8fb57a4bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:39.753265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:39.754410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:39.755228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:39.757132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:39.770454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:39.818514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc785061-aeac-4108-82c8-b3df0f344ca9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:39.837000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:39.837679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:39.837957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:39.838226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:39.838863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:39.840811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:39.870551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:39.902323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 834/1688] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:39 2022] POST /identity/v3/projects => generated 381 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:39.906467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:40.031366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:40.044598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:40.075409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 855/1689] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:40.100291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614da61c-517f-46c1-9fc5-5d634378e2b6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:40.120018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:40.120663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:40.120946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:40.129812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:40.130687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:40.136301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:40.211820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 835/1690] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:39 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:11:40.213156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:40.233522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9cb46-9296-4f7f-81eb-6942fe76ce84 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:40.246856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:40.247496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:40.248794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:40.249390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:40.255830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:40.267386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:40.288495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 856/1691] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:11:40.293967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:40.313205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9716ec-f9af-4a6b-9838-f06b1253e110 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:40.335116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd46c0e-8290-429c-8caf-c227610cbc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:40.340067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46c0e-8290-429c-8caf-c227610cbc85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:40.340336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46c0e-8290-429c-8caf-c227610cbc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:40.340619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46c0e-8290-429c-8caf-c227610cbc85 admin admin] PATH_INFO: `/v3/projects/bc362c533e80470dbde957dc167b7fda/users/e77914dbfcdb4d88b52c88b88b3e6ee3/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:40.425419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd46c0e-8290-429c-8caf-c227610cbc85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc362c533e80470dbde957dc167b7fda, user_id=e77914dbfcdb4d88b52c88b88b3e6ee3, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:40.432754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd46c0e-8290-429c-8caf-c227610cbc85 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:40.477629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 836/1692] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:40 2022] PUT /identity/v3/projects/bc362c533e80470dbde957dc167b7fda/users/e77914dbfcdb4d88b52c88b88b3e6ee3/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:40.485498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:40.499599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c062616-b96f-44e4-96ca-b799724768ce admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:40.639128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:40.680671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:40.681418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:40.681768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:40.682167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:40.682935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:40.684994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:40.702336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345e79d-a9f9-4b88-974c-eced76dbb5b1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:40.702717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345e79d-a9f9-4b88-974c-eced76dbb5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:40.703154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345e79d-a9f9-4b88-974c-eced76dbb5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:40.742016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 857/1693] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:40.742766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:40.751216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:40.815883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc9540-77c6-477c-8b96-60e2609b176a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:40.856191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:40.857509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f345e79d-a9f9-4b88-974c-eced76dbb5b1 None None] MFA Rules not processed for user `0354d80eb898418fae8acf8baf4894b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:40.858690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f345e79d-a9f9-4b88-974c-eced76dbb5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:40.861511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:40.862045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:40.862412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] PATH_INFO: `/v3/projects/b8d80cdbf2a642f7a5613a41e45fcb49` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:40.869664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d80cdbf2a642f7a5613a41e45fcb49)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:40.883261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:40.982111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d80cdbf2a642f7a5613a41e45fcb49', 'request_id': 'req-1e907990-fa21-466a-9572-6757cdfa25a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:40.995164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d80cdbf2a642f7a5613a41e45fcb49', 'request_id': 'req-1e907990-fa21-466a-9572-6757cdfa25a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:41.016488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 837/1694] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 06 01:11:41.016899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d80cdbf2a642f7a5613a41e45fcb49', 'request_id': 'req-1e907990-fa21-466a-9572-6757cdfa25a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:41.025176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345e79d-a9f9-4b88-974c-eced76dbb5b1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:41.041611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] The token cache is being invalidate because project b8d80cdbf2a642f7a5613a41e45fcb49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:41.041908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:41.046624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e907990-fa21-466a-9572-6757cdfa25a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:41.050028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 858/1695] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:40 2022] DELETE /identity/v3/projects/b8d80cdbf2a642f7a5613a41e45fcb49 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:41.051563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d3cae-dfa3-4c19-b22e-909848096f77 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:41.052951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345e79d-a9f9-4b88-974c-eced76dbb5b1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:41.053592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d3cae-dfa3-4c19-b22e-909848096f77 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:41.053955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d3cae-dfa3-4c19-b22e-909848096f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:41.100993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ea8d3cae-dfa3-4c19-b22e-909848096f77 None None] MFA Rules not processed for user `0354d80eb898418fae8acf8baf4894b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:41.102260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ea8d3cae-dfa3-4c19-b22e-909848096f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:41.197332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:41.251220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386263e4-a67b-4c82-91f7-3d6f008925ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:41.251965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386263e4-a67b-4c82-91f7-3d6f008925ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:41.252251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386263e4-a67b-4c82-91f7-3d6f008925ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:41.254161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386263e4-a67b-4c82-91f7-3d6f008925ea admin admin] PATH_INFO: `/v3/projects/bc362c533e80470dbde957dc167b7fda/users/e77914dbfcdb4d88b52c88b88b3e6ee3/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:41.265727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386263e4-a67b-4c82-91f7-3d6f008925ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc362c533e80470dbde957dc167b7fda, user_id=e77914dbfcdb4d88b52c88b88b3e6ee3, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:41.277196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386263e4-a67b-4c82-91f7-3d6f008925ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:41.333061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 838/1696] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:41 2022] PUT /identity/v3/projects/bc362c533e80470dbde957dc167b7fda/users/e77914dbfcdb4d88b52c88b88b3e6ee3/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:41.334562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b9f4f-f9e7-4fee-a2f2-071eae905ab5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:41.334852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b9f4f-f9e7-4fee-a2f2-071eae905ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:41.335130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b9f4f-f9e7-4fee-a2f2-071eae905ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:41.343423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 859/1697] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:41.344830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:41.345117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:41.345390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:41.422274 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1b1b9f4f-f9e7-4fee-a2f2-071eae905ab5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:41.423514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1b1b9f4f-f9e7-4fee-a2f2-071eae905ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:41.457473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] MFA Rules not processed for user `e77914dbfcdb4d88b52c88b88b3e6ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:41.458961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:41.642365 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:41.690477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 860/1698] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:41.691427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:41.696813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:41.706423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 839/1699] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:41.720048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:41.722189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:41.726085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:41.807941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] MFA Rules not processed for user `703464e4a9524efaa8fdfc1870158cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:41.810219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:41.941598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5b2ab-fd45-48b9-91fb-a0f986905319 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:41.967925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56289b6-ce95-49ab-acf5-60806d9b41fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:41.968677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56289b6-ce95-49ab-acf5-60806d9b41fa service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:41.969898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56289b6-ce95-49ab-acf5-60806d9b41fa service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:41.970202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56289b6-ce95-49ab-acf5-60806d9b41fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:41.974970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e56289b6-ce95-49ab-acf5-60806d9b41fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:42.015560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 840/1700] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:42.016596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:42.020756 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:42.207482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3192d0-3961-4d5d-9a98-9ad533db4604 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:42.219858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40146d1b-5655-4679-853e-89f62b74adf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:42.220738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146d1b-5655-4679-853e-89f62b74adf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.221910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146d1b-5655-4679-853e-89f62b74adf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.222906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56289b6-ce95-49ab-acf5-60806d9b41fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:42.223414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146d1b-5655-4679-853e-89f62b74adf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.225905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56289b6-ce95-49ab-acf5-60806d9b41fa service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:42.260533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146d1b-5655-4679-853e-89f62b74adf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:42.262142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146d1b-5655-4679-853e-89f62b74adf5 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:42.267034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 861/1701] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:42.268694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79372-f323-4428-8df4-80ee8b51b7e7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.269022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79372-f323-4428-8df4-80ee8b51b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.273004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79372-f323-4428-8df4-80ee8b51b7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.302413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 841/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:42.304136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a11cde-5982-4fae-ba74-99f533612c47 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.304136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a11cde-5982-4fae-ba74-99f533612c47 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.304324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a11cde-5982-4fae-ba74-99f533612c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.331834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cef79372-f323-4428-8df4-80ee8b51b7e7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:42.335988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cef79372-f323-4428-8df4-80ee8b51b7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:42.343466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-46a11cde-5982-4fae-ba74-99f533612c47 None None] MFA Rules not processed for user `703464e4a9524efaa8fdfc1870158cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:42.344735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-46a11cde-5982-4fae-ba74-99f533612c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:42.355245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 862/1703] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:42.357132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a546ab-5ff0-40e0-b757-4725fa3e4381 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.357417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a546ab-5ff0-40e0-b757-4725fa3e4381 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.357692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a546ab-5ff0-40e0-b757-4725fa3e4381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.379639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 842/1704] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:42.380751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a11cde-5982-4fae-ba74-99f533612c47 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:42.406196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a11cde-5982-4fae-ba74-99f533612c47 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:42.408159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-73a546ab-5ff0-40e0-b757-4725fa3e4381 None None] MFA Rules not processed for user `f8f44df824414e5ebc8cfd2126e9d66f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:42.411527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73a546ab-5ff0-40e0-b757-4725fa3e4381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:42.414960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:42.415306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.415588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.415889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.416997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:42.417314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:42.420121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:42.421930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:42.448058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 843/1705] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:42.449230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:42.452704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:42.492700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645d2c-47ea-4a8d-a261-d4866e02c9f1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:42.505372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:42.505981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.506259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.506529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.519576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:42.522057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:42.534186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 863/1706] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:42.540900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a546ab-5ff0-40e0-b757-4725fa3e4381 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:42.545304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 844/1707] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:42.585299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a546ab-5ff0-40e0-b757-4725fa3e4381 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:42.602649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:42.604220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.604500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.604768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.605474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:42.610767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:42.680593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 864/1708] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:42 2022] POST /identity/v3/projects => generated 403 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:42.684661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:42.709183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9a12-1e79-431b-9f7a-aa53e57262bd service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:42.930887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:42.932332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:42.932715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:42.933195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:42.934015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:42.937951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:42.983504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 845/1709] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:42 2022] POST /identity/v3/users => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:42.987464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.008679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fc174-ac1c-44fc-8df5-893f2efb4f80 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:43.015496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:43.016382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.016883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.017267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.018129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:43.020514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:43.031044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:43.050318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 865/1710] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:43.054448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.084810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3ed8-ca43-44eb-81e6-0959046d4e2c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:43.102453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f2b2e-3fca-4196-9bdb-ebe9ef958473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:43.109868 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f2b2e-3fca-4196-9bdb-ebe9ef958473 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.110543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f2b2e-3fca-4196-9bdb-ebe9ef958473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.110936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f2b2e-3fca-4196-9bdb-ebe9ef958473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.112488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f2b2e-3fca-4196-9bdb-ebe9ef958473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:43.114424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f2b2e-3fca-4196-9bdb-ebe9ef958473 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:43.135540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 846/1711] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:43.136979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.155212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454f89b-3a73-418d-8aae-f25265867fa8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:43.163263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:43.164714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.166011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.166527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] PATH_INFO: `/v3/projects/5176c0cb5fb4400987b1d7c77ceabab6/users/12998de4e3044171afcb7e5da42edef5/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.206801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5176c0cb5fb4400987b1d7c77ceabab6, user_id=12998de4e3044171afcb7e5da42edef5, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:43.216224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:43.249864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 866/1712] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:43 2022] PUT /identity/v3/projects/5176c0cb5fb4400987b1d7c77ceabab6/users/12998de4e3044171afcb7e5da42edef5/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:43.257091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9194023-48ff-48e5-b6c1-22388a6ee6e7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.257399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9194023-48ff-48e5-b6c1-22388a6ee6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.257680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9194023-48ff-48e5-b6c1-22388a6ee6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.270499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.340355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a9194023-48ff-48e5-b6c1-22388a6ee6e7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:43.353958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9194023-48ff-48e5-b6c1-22388a6ee6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.471728 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca10264-7094-42a9-b746-d077919adbbb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:43.483534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27c266-0d4d-4c2d-9295-0ee16a99ead3 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e77914dbfcdb4d88b52c88b88b3e6ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc362c533e80470dbde957dc167b7fda', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:43.484266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c266-0d4d-4c2d-9295-0ee16a99ead3 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.484560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c266-0d4d-4c2d-9295-0ee16a99ead3 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.486122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c266-0d4d-4c2d-9295-0ee16a99ead3 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.486828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c266-0d4d-4c2d-9295-0ee16a99ead3 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:43.487123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c266-0d4d-4c2d-9295-0ee16a99ead3 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:43.491706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c266-0d4d-4c2d-9295-0ee16a99ead3 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:43.495175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 847/1713] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:43.496642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf806b6-e78c-4334-8092-bbf6a67593a1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.496931 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf806b6-e78c-4334-8092-bbf6a67593a1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.497349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf806b6-e78c-4334-8092-bbf6a67593a1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.499188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 848/1714] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 6 01:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:43.499974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf806b6-e78c-4334-8092-bbf6a67593a1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.502189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5cf806b6-e78c-4334-8092-bbf6a67593a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.545105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 867/1715] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:43.545105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.545105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.545105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.558271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf806b6-e78c-4334-8092-bbf6a67593a1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:43.565635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:43.566878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.567342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.567810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] PATH_INFO: `/v3/users/928e347b25664d7891e45aa6066ba58f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.569445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=928e347b25664d7891e45aa6066ba58f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:43.572386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:43.628300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] MFA Rules not processed for user `bc47f8d295ed481a87b4fb0d405c605e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:43.629856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.694705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '928e347b25664d7891e45aa6066ba58f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:43.695520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '928e347b25664d7891e45aa6066ba58f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:43.713573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '928e347b25664d7891e45aa6066ba58f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:43.713573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:43.735375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '928e347b25664d7891e45aa6066ba58f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:43.736160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b1a4f8ac-fd33-41d4-bc97-14d4aacbcc75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '928e347b25664d7891e45aa6066ba58f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:43.743558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 868/1716] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:11:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:43.744684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.746291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.755349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 849/1717] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:43 2022] DELETE /identity/v3/users/928e347b25664d7891e45aa6066ba58f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:43.760298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe6058-7907-480c-aca5-90a829dbf7a0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.760688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe6058-7907-480c-aca5-90a829dbf7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.761082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe6058-7907-480c-aca5-90a829dbf7a0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.762689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 850/1718] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 6 01:11:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:43.762803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbce532-ebe8-44cb-b89c-06f434a2c6c2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:43.763388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe6058-7907-480c-aca5-90a829dbf7a0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.765277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1efe6058-7907-480c-aca5-90a829dbf7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.834242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe6058-7907-480c-aca5-90a829dbf7a0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:43.839856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:43.840740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.841229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.841689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.843869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.850124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca021c3a-c1cc-45ef-b422-e3363403d1c7 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e77914dbfcdb4d88b52c88b88b3e6ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc362c533e80470dbde957dc167b7fda', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:43.850758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021c3a-c1cc-45ef-b422-e3363403d1c7 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.851044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021c3a-c1cc-45ef-b422-e3363403d1c7 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.851361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca021c3a-c1cc-45ef-b422-e3363403d1c7 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.851987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca021c3a-c1cc-45ef-b422-e3363403d1c7 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:43.853956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca021c3a-c1cc-45ef-b422-e3363403d1c7 tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:43.870175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:43.897831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 869/1719] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:43 2022] POST /identity/v3/projects => generated 354 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:43.897831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13c12b-4845-4a45-9c73-37e7c3844a3b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:43.897831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13c12b-4845-4a45-9c73-37e7c3844a3b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:43.898891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13c12b-4845-4a45-9c73-37e7c3844a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:43.927159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4d13c12b-4845-4a45-9c73-37e7c3844a3b None None] MFA Rules not processed for user `2dc5c46544a447159b7fd3d225cd9697`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:43.928415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4d13c12b-4845-4a45-9c73-37e7c3844a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.942648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:43.945120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:43.960323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 851/1720] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:43.961554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:43.963034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:43.974590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fec3f-8b21-4209-8dbe-b58c96ec4e93 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:44.011612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 870/1721] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:44.011612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e1baa-7965-4d93-b938-b00bd0cc0996 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:44.011612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e1baa-7965-4d93-b938-b00bd0cc0996 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:44.011612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e1baa-7965-4d93-b938-b00bd0cc0996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:44.030522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4f2e1baa-7965-4d93-b938-b00bd0cc0996 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:44.031993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f2e1baa-7965-4d93-b938-b00bd0cc0996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:44.054915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3343a-c585-428d-98ae-696384c1784b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:44.055676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3343a-c585-428d-98ae-696384c1784b service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:44.056431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3343a-c585-428d-98ae-696384c1784b service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:44.056887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3343a-c585-428d-98ae-696384c1784b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:44.065469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3343a-c585-428d-98ae-696384c1784b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:44.067551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3343a-c585-428d-98ae-696384c1784b service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:44.078597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 852/1722] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:44.080282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb8d03-3052-4d45-ae54-0784306e8d7e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:44.080576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb8d03-3052-4d45-ae54-0784306e8d7e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:44.080851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb8d03-3052-4d45-ae54-0784306e8d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:44.101561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 871/1723] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:44.101561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e1baa-7965-4d93-b938-b00bd0cc0996 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:44.107565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-74cb8d03-3052-4d45-ae54-0784306e8d7e None None] MFA Rules not processed for user `bc47f8d295ed481a87b4fb0d405c605e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:44.109477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-74cb8d03-3052-4d45-ae54-0784306e8d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:44.128045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 853/1724] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:11:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:44.128999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb8d03-3052-4d45-ae54-0784306e8d7e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:44.151786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb8d03-3052-4d45-ae54-0784306e8d7e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:44.163191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:44.164386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:44.164750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:44.164914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:44.195832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:44.197856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:44.228290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 854/1725] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:44.231284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:44.242952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e1baa-7965-4d93-b938-b00bd0cc0996 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:44.255645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:44.257051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e77914dbfcdb4d88b52c88b88b3e6ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc362c533e80470dbde957dc167b7fda', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:44.257675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:44.257962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:44.258244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:44.258945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:44.259264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:44.260924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:44.269262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76a95e-90c1-46f5-b9f1-6ac4ad6be43b service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:44.282921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:44.283966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:44.284312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:44.284580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:44.285625 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 872/1726] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:44.298215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:44.300251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:44.310795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 855/1727] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:44.357946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:44.371119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef0de5-d02f-497b-a5cd-1331c895c44e tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:44.378036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e77914dbfcdb4d88b52c88b88b3e6ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc362c533e80470dbde957dc167b7fda', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:44.379040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:44.379179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:44.379400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] PATH_INFO: `/v3/projects/6a1789564a844cc4992d8fc10b28b1f8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:44.380874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6a1789564a844cc4992d8fc10b28b1f8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:44.383165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:44.426525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:44.454070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1789564a844cc4992d8fc10b28b1f8', 'request_id': 'req-bc224753-67b5-4ec1-8a5c-8db44b791efe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:44.462193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1789564a844cc4992d8fc10b28b1f8', 'request_id': 'req-bc224753-67b5-4ec1-8a5c-8db44b791efe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:44.478957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1789564a844cc4992d8fc10b28b1f8', 'request_id': 'req-bc224753-67b5-4ec1-8a5c-8db44b791efe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:44.516777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] The token cache is being invalidate because project 6a1789564a844cc4992d8fc10b28b1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:44.517392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:44.518006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:44.544114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 873/1728] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:44 2022] DELETE /identity/v3/projects/6a1789564a844cc4992d8fc10b28b1f8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:45.054256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:45.056132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:45.083411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fca72-1563-4a70-814b-1d8ed3330536 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:45.179423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:45.181508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:45.237952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:45.238713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:45.239088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:45.239593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] PATH_INFO: `/v3/users/660e79e294b241dd8087a8cfa2725f98` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:45.241286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] RBAC: Authorizing `identity:delete_user(user_id=660e79e294b241dd8087a8cfa2725f98)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:45.244488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:45.394048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '660e79e294b241dd8087a8cfa2725f98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:45.395406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '660e79e294b241dd8087a8cfa2725f98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:45.409307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc224753-67b5-4ec1-8a5c-8db44b791efe tempest-VolumeQuotaClassesTest-1470188401 tempest-VolumeQuotaClassesTest-1470188401-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:45.414418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '660e79e294b241dd8087a8cfa2725f98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:45.427301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a6f0a-95a3-4416-9684-aabea086a7f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:45.428213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a6f0a-95a3-4416-9684-aabea086a7f1 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:45.428504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a6f0a-95a3-4416-9684-aabea086a7f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:45.435875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a6f0a-95a3-4416-9684-aabea086a7f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:45.441036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1e3a6f0a-95a3-4416-9684-aabea086a7f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:45.468157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '660e79e294b241dd8087a8cfa2725f98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:45.469693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-76733d05-f50c-4da1-989e-a30ac9eb985b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '660e79e294b241dd8087a8cfa2725f98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:45.484220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a6f0a-95a3-4416-9684-aabea086a7f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:45.487025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a6f0a-95a3-4416-9684-aabea086a7f1 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:45.510135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 856/1729] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:45 2022] DELETE /identity/v3/users/660e79e294b241dd8087a8cfa2725f98 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:45.512106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4780e-dafc-4abf-9578-2edade922118 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:45.513497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 874/1730] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:45.514016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4780e-dafc-4abf-9578-2edade922118 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:45.514402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4780e-dafc-4abf-9578-2edade922118 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:45.517132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:45.517695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 857/1731] 10.222.0.23 () {54 vars in 961 bytes} [Tue Sep 6 01:11:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:45.518253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:45.518658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:45.525437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:45.525879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:45.526301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:45.580760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:45.581090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:45.585954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:45.587292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:45.747919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 875/1732] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:45.749203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:45.761320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:45.803252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0b5c2-3c8b-42c2-8113-d15d223bbf71 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:45.829187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:45.829947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:45.830280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:45.830614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] PATH_INFO: `/v3/projects/c24534055771473ebdf921acdbaf4b73` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:45.833444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24534055771473ebdf921acdbaf4b73)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:45.836250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:45.847631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 858/1733] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Sep 6 01:11:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:45.848655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:45.850889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:45.867494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:45.994915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:46.022994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24534055771473ebdf921acdbaf4b73', 'request_id': 'req-3c891bdb-b698-4021-b24a-621ee11cfa96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.041784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24534055771473ebdf921acdbaf4b73', 'request_id': 'req-3c891bdb-b698-4021-b24a-621ee11cfa96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.051701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad194-b57b-46bf-94e1-79c68cd96385 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:46.062434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24534055771473ebdf921acdbaf4b73', 'request_id': 'req-3c891bdb-b698-4021-b24a-621ee11cfa96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.097175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] The token cache is being invalidate because project c24534055771473ebdf921acdbaf4b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:46.097471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.114083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.128654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 876/1734] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:45 2022] DELETE /identity/v3/projects/c24534055771473ebdf921acdbaf4b73 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:46.132771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:46.140002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71019acb-0bf9-425f-80bd-0b4103c18b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:46.140552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:46.140640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71019acb-0bf9-425f-80bd-0b4103c18b39 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:46.140887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71019acb-0bf9-425f-80bd-0b4103c18b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:46.142235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71019acb-0bf9-425f-80bd-0b4103c18b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:46.144356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-71019acb-0bf9-425f-80bd-0b4103c18b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:46.274482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71019acb-0bf9-425f-80bd-0b4103c18b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:46.280839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71019acb-0bf9-425f-80bd-0b4103c18b39 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:46.311184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c891bdb-b698-4021-b24a-621ee11cfa96 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:46.333355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 859/1735] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:46.334900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9649f78-5ba4-4365-84eb-16e6f8f911f7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:46.335199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9649f78-5ba4-4365-84eb-16e6f8f911f7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:46.335324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:46.335471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9649f78-5ba4-4365-84eb-16e6f8f911f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:46.336083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:46.336264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:46.336546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:46.341876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:46.390541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:46.397587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:46.415497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 877/1736] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:46.417495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:46.420958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:46.443181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d6656-5643-4fa4-bb1e-d958acea5a6d service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:46.449666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:46.450493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:46.450906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:46.451330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] PATH_INFO: `/v3/users/1a39d55215274e7ea6d5431654e5f029` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:46.452724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a39d55215274e7ea6d5431654e5f029)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:46.455317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:46.486370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a9649f78-5ba4-4365-84eb-16e6f8f911f7 None None] MFA Rules not processed for user `12998de4e3044171afcb7e5da42edef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:46.492653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9649f78-5ba4-4365-84eb-16e6f8f911f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:46.563542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a39d55215274e7ea6d5431654e5f029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.589163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a39d55215274e7ea6d5431654e5f029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.634855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a39d55215274e7ea6d5431654e5f029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.636598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a39d55215274e7ea6d5431654e5f029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.659683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a39d55215274e7ea6d5431654e5f029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:46.681830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 878/1737] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:46 2022] DELETE /identity/v3/users/1a39d55215274e7ea6d5431654e5f029 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:46.768997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 860/1738] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:46.772057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:46.773680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:46.897917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:47.000821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7897d0f-0754-42ff-ae2f-9ca559303a7e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:47.014819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:47.015517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:47.015811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:47.016087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:47.016737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:47.023705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:47.123135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 879/1739] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:46 2022] POST /identity/v3/projects => generated 405 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:47.124827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9649f78-5ba4-4365-84eb-16e6f8f911f7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:47.132719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:47.156690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9649f78-5ba4-4365-84eb-16e6f8f911f7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:47.209558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:47.211508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:47.443924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:47.444850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:47.445290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:47.445709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:47.447194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:47.457744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:47.458759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70654a-113c-4c83-9213-21500fb2e0a8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:47.473887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:47.474859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:47.475366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:47.475877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] PATH_INFO: `/v3/projects/80d68779ebb94b8cbe0f142391e62216` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:47.479100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d68779ebb94b8cbe0f142391e62216)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:47.482530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:47.573979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 861/1740] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:47 2022] POST /identity/v3/users => generated 309 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:11:47.579173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:47.593950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d68779ebb94b8cbe0f142391e62216', 'request_id': 'req-ae54bcb1-f977-4274-a4d4-98c60db89a7c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:47.637640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ba71f-4621-4cfb-b2b5-4b37b586b9f2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:47.649481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d68779ebb94b8cbe0f142391e62216', 'request_id': 'req-ae54bcb1-f977-4274-a4d4-98c60db89a7c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:47.695366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d68779ebb94b8cbe0f142391e62216', 'request_id': 'req-ae54bcb1-f977-4274-a4d4-98c60db89a7c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:47.758463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] The token cache is being invalidate because project 80d68779ebb94b8cbe0f142391e62216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:47.765118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:47.765620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:47.772254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 880/1741] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:47 2022] DELETE /identity/v3/projects/80d68779ebb94b8cbe0f142391e62216 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:11:47.773319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:47.782127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:47.888050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:47.905916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:47.910977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:47.913210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:47.913505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:47.914296 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:47.924368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:47.929491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54bcb1-f977-4274-a4d4-98c60db89a7c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:47.948704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:47.949722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:47.951184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:47.951907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] PATH_INFO: `/v3/projects/f9f86fe3a6b4468da862006aa4cbe4f1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:47.954954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f86fe3a6b4468da862006aa4cbe4f1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:47.960110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:47.996132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 862/1742] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:48.004428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:48.006419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:48.044017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a879c-ebb4-4e6c-bca7-1a2018e325e1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:48.053632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:48.076906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:48.083036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:48.084548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] PATH_INFO: `/v3/users/e77914dbfcdb4d88b52c88b88b3e6ee3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:48.088951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77914dbfcdb4d88b52c88b88b3e6ee3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:48.112284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:48.146343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f86fe3a6b4468da862006aa4cbe4f1', 'request_id': 'req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.156174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f86fe3a6b4468da862006aa4cbe4f1', 'request_id': 'req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.170314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f86fe3a6b4468da862006aa4cbe4f1', 'request_id': 'req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.207370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] The token cache is being invalidate because project f9f86fe3a6b4468da862006aa4cbe4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:48.207878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.210589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.220422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 881/1743] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:47 2022] DELETE /identity/v3/projects/f9f86fe3a6b4468da862006aa4cbe4f1 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:48.221803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:48.230997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:48.323095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77914dbfcdb4d88b52c88b88b3e6ee3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.324410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77914dbfcdb4d88b52c88b88b3e6ee3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.339526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77914dbfcdb4d88b52c88b88b3e6ee3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.365986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77914dbfcdb4d88b52c88b88b3e6ee3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.367252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f863c71d-91a8-4944-b51d-bbf554f39526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77914dbfcdb4d88b52c88b88b3e6ee3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:48.382679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:48.429979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 863/1744] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:47 2022] DELETE /identity/v3/users/e77914dbfcdb4d88b52c88b88b3e6ee3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:48.435744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085e9e-1612-4bc0-9026-6587a5e1f21d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:48.436315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085e9e-1612-4bc0-9026-6587a5e1f21d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:48.439555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085e9e-1612-4bc0-9026-6587a5e1f21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:48.555127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ef085e9e-1612-4bc0-9026-6587a5e1f21d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:48.556960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ef085e9e-1612-4bc0-9026-6587a5e1f21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:48.582247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb119d-0d71-4e02-8f08-6696b67a1f2e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:48.838758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa64f3-3f04-4d43-8e3a-bdf85d80284e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:48.838758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa64f3-3f04-4d43-8e3a-bdf85d80284e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:48.838758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa64f3-3f04-4d43-8e3a-bdf85d80284e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:48.838758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa64f3-3f04-4d43-8e3a-bdf85d80284e admin admin] PATH_INFO: `/v3/projects/e3e7749c4f0a4e7bb81a3b5278a2ba8d/users/c31c86da27064c7a939dc6a3adc427b6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:48.930112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 864/1745] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:48.935803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a713-70c3-4fbb-9bbf-97278d7eb735 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:48.938602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a713-70c3-4fbb-9bbf-97278d7eb735 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:48.938932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a713-70c3-4fbb-9bbf-97278d7eb735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:48.947462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa64f3-3f04-4d43-8e3a-bdf85d80284e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e7749c4f0a4e7bb81a3b5278a2ba8d, user_id=c31c86da27064c7a939dc6a3adc427b6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:48.950210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa64f3-3f04-4d43-8e3a-bdf85d80284e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:49.006638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 882/1746] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:48 2022] PUT /identity/v3/projects/e3e7749c4f0a4e7bb81a3b5278a2ba8d/users/c31c86da27064c7a939dc6a3adc427b6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:49.008116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:49.008408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:49.008684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:49.036805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7e37a713-70c3-4fbb-9bbf-97278d7eb735 None None] MFA Rules not processed for user `703464e4a9524efaa8fdfc1870158cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:49.038255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:49.043063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7e37a713-70c3-4fbb-9bbf-97278d7eb735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:49.066997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:49.071993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:49.281460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 883/1747] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:49.282421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:49.285865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:49.294600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 865/1748] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:49.296314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a713-70c3-4fbb-9bbf-97278d7eb735 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:49.336780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a713-70c3-4fbb-9bbf-97278d7eb735 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:49.346763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c3b2-e162-4107-a47f-b24f9accaea0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:49.350941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:49.357194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:49.357512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:49.367341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:49.368804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:49.371615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:49.373491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:49.373491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:49.374208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:49.375275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:49.377091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:49.381943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:49.382668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:49.422225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 884/1749] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:49.440877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 866/1750] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:11:49.444904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:49.445137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:49.467428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ad212-7a89-4b05-b3e5-1ff0c37f80d8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:49.477300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:49.478291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:49.478715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:49.479140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] PATH_INFO: `/v3/projects/e3e7749c4f0a4e7bb81a3b5278a2ba8d/users/c31c86da27064c7a939dc6a3adc427b6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:49.486167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcda789-790e-4f1a-8798-6430a572d719 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:49.495740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e7749c4f0a4e7bb81a3b5278a2ba8d, user_id=c31c86da27064c7a939dc6a3adc427b6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:49.499123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:49.503896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:49.504745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:49.505687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:49.506318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:49.507199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:49.517869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:49.550265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 885/1751] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:49 2022] PUT /identity/v3/projects/e3e7749c4f0a4e7bb81a3b5278a2ba8d/users/c31c86da27064c7a939dc6a3adc427b6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:49.625897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 867/1752] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:49 2022] POST /identity/v3/projects => generated 399 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:49.627168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:49.890180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce48ff-f277-4f4c-b8c6-855fc4a8ae46 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:50.024063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:50.026250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:50.110801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:50.114170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.116644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.117163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:50.121026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:50.123561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:50.223896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 886/1753] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:49 2022] POST /identity/v3/users => generated 300 bytes in 594 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:50.225154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:50.278429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461791-15b4-43bd-a82f-ba20f839f9f9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:50.300172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:50.307767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.308157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.308490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:50.311630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:50.313368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:50.333419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d9f87-19e3-4967-9164-042c6d068832 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:50.351238 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:50.361208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.362279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.362781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] PATH_INFO: `/v3/projects/bc362c533e80470dbde957dc167b7fda` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:50.366351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc362c533e80470dbde957dc167b7fda)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:50.373678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:50.381583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 887/1754] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:50.382995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:50.418817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f13be-23b4-4aef-84c4-b99d9cd54ff0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:50.434844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:50.435703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.436305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.437011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:50.438391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:50.444842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:50.447770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:50.451445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:50.483049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 888/1755] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:50.484328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:50.514255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441072-453d-4ffa-8f88-272f22e4472e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:50.533265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c4fe1-ae4d-45b8-9a80-c6ad2bb93bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:50.534120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4fe1-ae4d-45b8-9a80-c6ad2bb93bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.534661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4fe1-ae4d-45b8-9a80-c6ad2bb93bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.535157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c4fe1-ae4d-45b8-9a80-c6ad2bb93bd3 admin admin] PATH_INFO: `/v3/projects/8a93214740dc4e4483afdab2a92cd9af/users/9784d72253ce4204972fdcbac12ea64d/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:50.563364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc362c533e80470dbde957dc167b7fda', 'request_id': 'req-2b8a6926-4b7e-4035-becc-222e3140ecad'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:50.569848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc362c533e80470dbde957dc167b7fda', 'request_id': 'req-2b8a6926-4b7e-4035-becc-222e3140ecad'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:50.585527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc362c533e80470dbde957dc167b7fda', 'request_id': 'req-2b8a6926-4b7e-4035-becc-222e3140ecad'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:50.596643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c4fe1-ae4d-45b8-9a80-c6ad2bb93bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a93214740dc4e4483afdab2a92cd9af, user_id=9784d72253ce4204972fdcbac12ea64d, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:50.604041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c4fe1-ae4d-45b8-9a80-c6ad2bb93bd3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:50.608448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] The token cache is being invalidate because project bc362c533e80470dbde957dc167b7fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:50.608746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:50.612743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8a6926-4b7e-4035-becc-222e3140ecad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:50.618068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 868/1756] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:50 2022] DELETE /identity/v3/projects/bc362c533e80470dbde957dc167b7fda => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:50.618068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ca004-771d-4220-944c-7d38ed7579d2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.618068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ca004-771d-4220-944c-7d38ed7579d2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.618068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ca004-771d-4220-944c-7d38ed7579d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:50.667675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 889/1757] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:50 2022] PUT /identity/v3/projects/8a93214740dc4e4483afdab2a92cd9af/users/9784d72253ce4204972fdcbac12ea64d/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:50.680898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab340c2-2bb9-4f4b-9365-70bf5755bb19 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.681186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab340c2-2bb9-4f4b-9365-70bf5755bb19 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.684454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab340c2-2bb9-4f4b-9365-70bf5755bb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:50.735433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9ab340c2-2bb9-4f4b-9365-70bf5755bb19 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:50.738119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9ab340c2-2bb9-4f4b-9365-70bf5755bb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:50.751026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a00ca004-771d-4220-944c-7d38ed7579d2 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:50.756241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a00ca004-771d-4220-944c-7d38ed7579d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:50.957095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 890/1758] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:50.970573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:50.972132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:50.983552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:51.002277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 869/1759] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:11:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:51.005662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191ebf-3235-4d01-b5ef-e55d7308cc96 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:51.010175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191ebf-3235-4d01-b5ef-e55d7308cc96 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:51.010175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd191ebf-3235-4d01-b5ef-e55d7308cc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:51.061015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] MFA Rules not processed for user `9784d72253ce4204972fdcbac12ea64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:51.062596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:51.063019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dd191ebf-3235-4d01-b5ef-e55d7308cc96 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:51.064483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dd191ebf-3235-4d01-b5ef-e55d7308cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:51.079250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 870/1760] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:51.080218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191ebf-3235-4d01-b5ef-e55d7308cc96 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:51.093506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd191ebf-3235-4d01-b5ef-e55d7308cc96 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:51.099030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:51.099927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:51.100897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:51.101261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:51.102200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:11:51.102617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:51.104338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:51.121093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 871/1761] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:11:51.127658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:51.246422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a575e2-ddc9-4c6d-ae71-d1b6af0b5eaf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:51.267250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 891/1762] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:51.271841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:51.277218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:51.277218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:51.277603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:51.277862 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:51.278250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:51.279004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:51.289406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:51.352607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c084ee4-63a2-44c7-98f5-9e77ab65e7ea None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:51.370795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:51.376933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:51.377204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:51.377485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] PATH_INFO: `/v3/users/ba1ba449c2d8465ba93c8091ae5eb028` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:51.379724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba1ba449c2d8465ba93c8091ae5eb028)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:51.390638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:51.398361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 872/1763] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:51 2022] POST /identity/v3/projects => generated 393 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:51.398361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:51.404981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:51.483179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:51.518871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba1ba449c2d8465ba93c8091ae5eb028'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:51.539139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba1ba449c2d8465ba93c8091ae5eb028'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:51.554449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:51.590541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51606532-d6f1-4a74-93bd-1c6d8d1c710f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:51.602929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba1ba449c2d8465ba93c8091ae5eb028'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:51.606608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba1ba449c2d8465ba93c8091ae5eb028'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:51.609485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:51.614454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:51.614852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:51.615135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:51.615845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:51.617811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:51.637294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba1ba449c2d8465ba93c8091ae5eb028'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:51.665660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 892/1764] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:51 2022] DELETE /identity/v3/users/ba1ba449c2d8465ba93c8091ae5eb028 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:51.666652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:51.668214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:51.705437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 873/1765] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:51 2022] POST /identity/v3/projects => generated 399 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:51.713751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:51.722073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:51.976012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008cd9a-74f0-4ffd-9383-221046c393af admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:51.994528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773a7ae-5f05-46fc-9649-a38c8bcb3af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:51.995290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773a7ae-5f05-46fc-9649-a38c8bcb3af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:51.995573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773a7ae-5f05-46fc-9649-a38c8bcb3af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:51.996526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773a7ae-5f05-46fc-9649-a38c8bcb3af5 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:51.997180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773a7ae-5f05-46fc-9649-a38c8bcb3af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.000057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773a7ae-5f05-46fc-9649-a38c8bcb3af5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.007885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f4c28-172f-45dc-affd-302a4bf25a5f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.070983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 874/1766] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:51 2022] POST /identity/v3/users => generated 297 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:52.077097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.081352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.081352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.157472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:52.158868 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:52.217877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 875/1767] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:52.221239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.226710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:52.290485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.291448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.292045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.292389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.296525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:52.297881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172b34a-4872-4ec9-b339-e3e1447fe10a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.308271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.310596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.310881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.311159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.311867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.318330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.376631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 876/1768] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:52 2022] POST /identity/v3/users => generated 306 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:52.379512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.413784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f031db-7092-470c-8d88-70c1448b3bcd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.427304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.430184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.430509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.432127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.432850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.437028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.465746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 877/1769] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:52.466753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.468945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:52.474097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.476021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.481573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:52.501401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 893/1770] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:11:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 835 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:11:52.502407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.518967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ef84a-f0de-452e-b718-a29a29ce70b6 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.534917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.536094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.536386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.536705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.537563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.539302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.572763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 894/1771] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:52.582279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.605659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be2666-70f8-4fc1-b098-d9b1e64e65ba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.623255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.624093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.624376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.624669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.625374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.626669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c3c54-1902-4586-91de-8af34e9f3c9b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.627300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.638078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.639088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.639583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.640099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.647308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 895/1772] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:52.652639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.664287 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.666874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.694061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 878/1773] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:52.701276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b996-2df3-49ac-a37c-1d5157f3a222 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.709303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.719890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.721553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.721863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.722378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] PATH_INFO: `/v3/projects/39f5e1c8e8604bdf9763ac9094698b59/users/6c48c0ac0a9d45cda7e8b710f8c4de01/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.734429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91374033-3e05-4907-be5e-a360aa308d3d service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:52.748250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fffa2-ff59-4fcd-9632-213bbe354bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:52.749009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fffa2-ff59-4fcd-9632-213bbe354bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.749363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fffa2-ff59-4fcd-9632-213bbe354bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.749710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fffa2-ff59-4fcd-9632-213bbe354bfc admin admin] PATH_INFO: `/v3/projects/8f4e50f1aa0c40a6b2851d5c6045e16d/users/17bc818a7cfe4e728373da3d6473e019/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.779694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:52.797502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fffa2-ff59-4fcd-9632-213bbe354bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4e50f1aa0c40a6b2851d5c6045e16d, user_id=17bc818a7cfe4e728373da3d6473e019, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.800898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fffa2-ff59-4fcd-9632-213bbe354bfc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.803191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f5e1c8e8604bdf9763ac9094698b59, user_id=6c48c0ac0a9d45cda7e8b710f8c4de01, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:52.813525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:52.831740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 879/1774] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:52 2022] PUT /identity/v3/projects/8f4e50f1aa0c40a6b2851d5c6045e16d/users/17bc818a7cfe4e728373da3d6473e019/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:52.844865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98c08-9eff-47dc-a2a5-bbc87b6bed3e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:52.845795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98c08-9eff-47dc-a2a5-bbc87b6bed3e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:52.845795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98c08-9eff-47dc-a2a5-bbc87b6bed3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:52.894185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 896/1775] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:52 2022] PUT /identity/v3/projects/39f5e1c8e8604bdf9763ac9094698b59/users/6c48c0ac0a9d45cda7e8b710f8c4de01/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:52.898375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:52.903176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c9b98c08-9eff-47dc-a2a5-bbc87b6bed3e None None] MFA Rules not processed for user `17bc818a7cfe4e728373da3d6473e019`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:52.904558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c9b98c08-9eff-47dc-a2a5-bbc87b6bed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:53.088265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 880/1776] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:11:53.089752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194d27-a2c7-4fd6-8b37-e6be0e3b96e7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.090190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194d27-a2c7-4fd6-8b37-e6be0e3b96e7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.090473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39194d27-a2c7-4fd6-8b37-e6be0e3b96e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.116746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24380fe-47b6-475b-a60f-14772bc6b82d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:53.136535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:53.137164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.137444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.137717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.138367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:53.138685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-39194d27-a2c7-4fd6-8b37-e6be0e3b96e7 None None] MFA Rules not processed for user `c31c86da27064c7a939dc6a3adc427b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:53.140083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:53.140705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-39194d27-a2c7-4fd6-8b37-e6be0e3b96e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:53.169561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 897/1777] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:53.184947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:53.228425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cceef-ef67-434e-8838-cf880c2b84a4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:53.244961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:53.246435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.246867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.247277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.256027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:53.263429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:53.285749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 881/1778] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:53 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:53.286897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39194d27-a2c7-4fd6-8b37-e6be0e3b96e7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:53.324495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39194d27-a2c7-4fd6-8b37-e6be0e3b96e7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:53.338132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6496ed4-db2f-4a65-8907-383caf676f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:53.339239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6496ed4-db2f-4a65-8907-383caf676f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.340044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6496ed4-db2f-4a65-8907-383caf676f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.343166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6496ed4-db2f-4a65-8907-383caf676f79 admin admin] PATH_INFO: `/v3/projects/39f5e1c8e8604bdf9763ac9094698b59/users/6c48c0ac0a9d45cda7e8b710f8c4de01/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.347801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6496ed4-db2f-4a65-8907-383caf676f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f5e1c8e8604bdf9763ac9094698b59, user_id=6c48c0ac0a9d45cda7e8b710f8c4de01, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:53.359459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6496ed4-db2f-4a65-8907-383caf676f79 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:53.371857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 898/1779] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:11:53 2022] POST /identity/v3/projects => generated 391 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:11:53.375803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:53.410251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 882/1780] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:53 2022] PUT /identity/v3/projects/39f5e1c8e8604bdf9763ac9094698b59/users/6c48c0ac0a9d45cda7e8b710f8c4de01/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:53.412059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.413834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.414419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.499902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] MFA Rules not processed for user `6c48c0ac0a9d45cda7e8b710f8c4de01`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:53.501273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:53.597864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5696597-7256-486a-9430-5a052a380a3d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:53.623569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:53.679895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 883/1781] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:53 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:53.682555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:53.684875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:53.707118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:53.776989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:53.777637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.777917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.778198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.778868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:53.781632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:53.802809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f6f41-4f43-4b6d-9538-7b73a72c7460 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:53.809036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:53.810073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.811406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.812316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.820262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 899/1782] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:11:53 2022] POST /identity/v3/users => generated 302 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:11:53.825359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:53.834633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:53.836848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:53.852114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ce126-2351-49bc-8f4f-b02ab094f90a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:53.859308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 884/1783] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:53 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:53.894128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:53.894858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.895316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.895608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.896203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:53.897819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:53.942469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 900/1784] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:11:53.943884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:53.964704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d971-e0b8-4e22-b2a0-75498b02857b service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:53.975875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:53.978934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:53.988339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:53.988991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] PATH_INFO: `/v3/projects/4cc9f85edece4529b67136e8b50d1422/users/70f6102f3c98410eb195895e1a076d4d/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:53.994004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:54.021080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344295ce-7a74-4029-a530-f5009048e420 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:54.025271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:54.025892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:54.026200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:54.026450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:54.032841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc9f85edece4529b67136e8b50d1422, user_id=70f6102f3c98410eb195895e1a076d4d, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:54.037433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:54.062776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:54.096957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 885/1785] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:53 2022] PUT /identity/v3/projects/4cc9f85edece4529b67136e8b50d1422/users/70f6102f3c98410eb195895e1a076d4d/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:54.098127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:54.170884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:54.172911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:54.211383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 901/1786] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:11:54.251205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801906cb-9aee-48d0-aecd-d9e0b2342cf3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:54.259618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637ddcc-773b-4f56-87e8-794cbf13d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:54.260511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637ddcc-773b-4f56-87e8-794cbf13d605 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:54.260905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637ddcc-773b-4f56-87e8-794cbf13d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:54.261302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637ddcc-773b-4f56-87e8-794cbf13d605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:54.262069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637ddcc-773b-4f56-87e8-794cbf13d605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:54.264924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637ddcc-773b-4f56-87e8-794cbf13d605 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:54.283454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 886/1787] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:11:54.288990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:54.307817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76cbce-d70f-465d-b42d-e05f4e9c8190 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:54.330820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:54.332696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:54.332981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:54.333308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] PATH_INFO: `/v3/projects/4cc9f85edece4529b67136e8b50d1422/users/70f6102f3c98410eb195895e1a076d4d/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:54.343945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc9f85edece4529b67136e8b50d1422, user_id=70f6102f3c98410eb195895e1a076d4d, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:54.345103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:54.373977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be393fc-77a0-46fb-8fc8-27e04c840360 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:54.374330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be393fc-77a0-46fb-8fc8-27e04c840360 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:54.375403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be393fc-77a0-46fb-8fc8-27e04c840360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:54.378658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 902/1788] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:11:54 2022] PUT /identity/v3/projects/4cc9f85edece4529b67136e8b50d1422/users/70f6102f3c98410eb195895e1a076d4d/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:54.429784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0be393fc-77a0-46fb-8fc8-27e04c840360 None None] MFA Rules not processed for user `70f6102f3c98410eb195895e1a076d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:54.431161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0be393fc-77a0-46fb-8fc8-27e04c840360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:54.624853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:54.717881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 887/1789] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:11:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:54.725853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:54.728277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:54.922783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef267d7-b814-44c4-ba80-d9568da531d3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:54.931585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede77009-0acd-46ca-bc27-2e74e33e1a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:54.932275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77009-0acd-46ca-bc27-2e74e33e1a66 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:54.932557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77009-0acd-46ca-bc27-2e74e33e1a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:54.935929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77009-0acd-46ca-bc27-2e74e33e1a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:54.960849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede77009-0acd-46ca-bc27-2e74e33e1a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:54.969787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede77009-0acd-46ca-bc27-2e74e33e1a66 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:54.999577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 903/1790] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:11:55.141401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be393fc-77a0-46fb-8fc8-27e04c840360 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:55.177951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be393fc-77a0-46fb-8fc8-27e04c840360 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:55.183485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:55.186369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:55.186917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:55.187441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:55.283517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:55.285277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:55.325720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 888/1791] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:56.566311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:56.566825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:56.566921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:56.621211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] MFA Rules not processed for user `bc47f8d295ed481a87b4fb0d405c605e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:56.622580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:56.743827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:56.846860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 904/1792] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 6 01:11:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:56.990851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:57.015508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c4666-9f6e-43f0-be39-f15b29f7506b service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:57.027886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:57.032007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:57.032896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:57.034106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:57.062559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:57.064522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:57.094477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 889/1793] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:57.124640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:57.126394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:57.397442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed48f-65ea-4e67-a88a-5302e80cd88f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:57.421959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:57.428579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:57.429532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:57.430800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] PATH_INFO: `/v3/projects/251dafdc7ff74ee98d6d76c540d28e49` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:57.435124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=251dafdc7ff74ee98d6d76c540d28e49)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:57.448696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:57.593972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251dafdc7ff74ee98d6d76c540d28e49', 'request_id': 'req-54c44852-151a-491d-9330-6008c5874e5d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.601772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251dafdc7ff74ee98d6d76c540d28e49', 'request_id': 'req-54c44852-151a-491d-9330-6008c5874e5d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.612229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251dafdc7ff74ee98d6d76c540d28e49', 'request_id': 'req-54c44852-151a-491d-9330-6008c5874e5d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.647935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] The token cache is being invalidate because project 251dafdc7ff74ee98d6d76c540d28e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:11:57.648245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.648924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-54c44852-151a-491d-9330-6008c5874e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.652375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 905/1794] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:11:57 2022] DELETE /identity/v3/projects/251dafdc7ff74ee98d6d76c540d28e49 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:57.657142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:57.679030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e194737-b198-417b-bdd7-46e31cb6aeca service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:57.830741 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:57.831663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:57.832064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:57.832487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] PATH_INFO: `/v3/users/08a9eb114eb14b35a22e3ea4160552c6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:57.836338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a9eb114eb14b35a22e3ea4160552c6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:57.840956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:57.954973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a9eb114eb14b35a22e3ea4160552c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.955862 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a9eb114eb14b35a22e3ea4160552c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.969877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a9eb114eb14b35a22e3ea4160552c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:57.987845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:11:57.996132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a9eb114eb14b35a22e3ea4160552c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:58.001396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bc134a8-8517-4e01-8adc-24bd2e927866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a9eb114eb14b35a22e3ea4160552c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:11:58.053316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 890/1795] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:11:57 2022] DELETE /identity/v3/users/08a9eb114eb14b35a22e3ea4160552c6 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:11:59.233518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf86f6-b660-41c9-b567-cc860a9f6d52 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:59.234183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf86f6-b660-41c9-b567-cc860a9f6d52 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:59.234183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf86f6-b660-41c9-b567-cc860a9f6d52 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:59.237763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 906/1796] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:11:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:11:59.403530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911e9cf-54b6-4df2-bb41-3f8c2db5dd60 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:59.404878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911e9cf-54b6-4df2-bb41-3f8c2db5dd60 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:59.405564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911e9cf-54b6-4df2-bb41-3f8c2db5dd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:59.452817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f911e9cf-54b6-4df2-bb41-3f8c2db5dd60 None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:11:59.455174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f911e9cf-54b6-4df2-bb41-3f8c2db5dd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:59.641245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 891/1797] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:11:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:11:59.690807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf86f6-b660-41c9-b567-cc860a9f6d52 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:11:59.693514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4abf86f6-b660-41c9-b567-cc860a9f6d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:11:59.745994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf86f6-b660-41c9-b567-cc860a9f6d52 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:11:59.775826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f2f0-b6e3-4ba5-9d58-27088bc3aa40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:11:59.777416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f2f0-b6e3-4ba5-9d58-27088bc3aa40 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:59.780012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f2f0-b6e3-4ba5-9d58-27088bc3aa40 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:59.780522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f2f0-b6e3-4ba5-9d58-27088bc3aa40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:59.801490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f2f0-b6e3-4ba5-9d58-27088bc3aa40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:11:59.804233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f2f0-b6e3-4ba5-9d58-27088bc3aa40 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:11:59.821001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 907/1798] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:11:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:11:59.980856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafcf61-8a01-4053-a744-2bd1abdd168d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:11:59.981142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafcf61-8a01-4053-a744-2bd1abdd168d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:11:59.981602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafcf61-8a01-4053-a744-2bd1abdd168d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:11:59.983235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 892/1799] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:12:00.019948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:00.020454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:00.020884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:00.116961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:00.118469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:00.305692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:00.328561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 908/1800] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:12:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:00.498397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafcf61-8a01-4053-a744-2bd1abdd168d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:00.500454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dcafcf61-8a01-4053-a744-2bd1abdd168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:00.550701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:00.552301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafcf61-8a01-4053-a744-2bd1abdd168d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:00.552789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:00.571485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:00.573399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:00.573399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:00.573399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:00.591262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:00.594028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:00.633019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 893/1801] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:00.636637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:00.640040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:00.830883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538b336-6a43-4cf9-b201-1a2be1f1ac3a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:00.843560 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:00.844247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:00.844532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:00.844809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] PATH_INFO: `/v3/users/17bc818a7cfe4e728373da3d6473e019` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:00.855179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=17bc818a7cfe4e728373da3d6473e019)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:00.858207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:00.893855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137053f7-e203-49fd-93df-b29e1ed6d6b2 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:00.908231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:00.908819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:00.909110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:00.909384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] PATH_INFO: `/v3/users/9784d72253ce4204972fdcbac12ea64d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:00.924807 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9784d72253ce4204972fdcbac12ea64d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:00.926889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:00.988904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17bc818a7cfe4e728373da3d6473e019'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.007693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17bc818a7cfe4e728373da3d6473e019'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.012961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9784d72253ce4204972fdcbac12ea64d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.014208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9784d72253ce4204972fdcbac12ea64d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.027571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9784d72253ce4204972fdcbac12ea64d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.038785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17bc818a7cfe4e728373da3d6473e019'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.040997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17bc818a7cfe4e728373da3d6473e019'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.042118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:01.052424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67cda989-1442-414d-a611-6c144a8ddb9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17bc818a7cfe4e728373da3d6473e019'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.052728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9784d72253ce4204972fdcbac12ea64d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.056965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b842918-c7cb-4472-b22a-ed4354163964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9784d72253ce4204972fdcbac12ea64d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.068103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 909/1802] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:00 2022] DELETE /identity/v3/users/17bc818a7cfe4e728373da3d6473e019 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:01.074856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:01.075403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:01.076060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:01.080749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 894/1803] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:00 2022] DELETE /identity/v3/users/9784d72253ce4204972fdcbac12ea64d => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:01.094057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:01.094519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:01.094950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:01.114222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:01.123833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:01.161614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:01.163812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:01.234266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 910/1804] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:01.296249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:01.298235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:01.327995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 895/1805] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:01.329152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ace0d2-f506-45a9-9e96-c1328448ab61 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:01.347492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:01.348000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:01.348945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:01.349959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:01.350465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:01.353258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] PATH_INFO: `/v3/projects/bb90bff777894d16a793ad4bcf508074` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:01.357587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb90bff777894d16a793ad4bcf508074)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:01.365773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:01.423528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:01.499044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d3146-6159-4894-a8ed-f5891c0009ef None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:01.545477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb90bff777894d16a793ad4bcf508074', 'request_id': 'req-3a8a395f-4fba-48af-b98b-22d85f0c1a98'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.565418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb90bff777894d16a793ad4bcf508074', 'request_id': 'req-3a8a395f-4fba-48af-b98b-22d85f0c1a98'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.592531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb90bff777894d16a793ad4bcf508074', 'request_id': 'req-3a8a395f-4fba-48af-b98b-22d85f0c1a98'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.612429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] The token cache is being invalidate because project bb90bff777894d16a793ad4bcf508074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:01.613770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.615037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:01.617752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 911/1806] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:01 2022] DELETE /identity/v3/projects/bb90bff777894d16a793ad4bcf508074 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:01.618739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:01.620646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:01.664998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da772bc-1070-4bf9-9658-b286e7524689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:01.665908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da772bc-1070-4bf9-9658-b286e7524689 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:01.666344 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da772bc-1070-4bf9-9658-b286e7524689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:01.667460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da772bc-1070-4bf9-9658-b286e7524689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:01.672518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3da772bc-1070-4bf9-9658-b286e7524689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:01.791676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a395f-4fba-48af-b98b-22d85f0c1a98 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:01.810524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:01.814608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:01.817526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:01.819414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:01.826915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:01.858569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:01.892769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da772bc-1070-4bf9-9658-b286e7524689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:01.894507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da772bc-1070-4bf9-9658-b286e7524689 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:01.926806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 896/1807] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:01.928033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef2df9-d18e-4e7d-a5f7-f3970b65b510 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:01.928313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef2df9-d18e-4e7d-a5f7-f3970b65b510 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:01.928592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef2df9-d18e-4e7d-a5f7-f3970b65b510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.024563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:02.026431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:02.028936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-50ef2df9-d18e-4e7d-a5f7-f3970b65b510 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:02.031012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50ef2df9-d18e-4e7d-a5f7-f3970b65b510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:02.041855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 897/1808] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:02.051943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4abdec-0d04-482d-a4b6-3378797ebb25 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.053822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4abdec-0d04-482d-a4b6-3378797ebb25 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.054398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4abdec-0d04-482d-a4b6-3378797ebb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.072423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 912/1809] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:02.086082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4f4abdec-0d04-482d-a4b6-3378797ebb25 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:02.088891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4f4abdec-0d04-482d-a4b6-3378797ebb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:02.101010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 898/1810] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:02.105654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:02.128472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65650825-cead-47de-98df-ffa6ea016cc3 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:02.141340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:02.142342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.142810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.143227 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.144185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:02.144688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:02.155612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:02.201092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 913/1811] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:02.202442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4abdec-0d04-482d-a4b6-3378797ebb25 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:02.225975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4abdec-0d04-482d-a4b6-3378797ebb25 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:02.243600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:02.244409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.244619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.244917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.248151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:02.254231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:02.330114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 899/1812] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:02 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:02.334258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:02.353754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39a436-da2b-4e4a-b570-3aba144b97cd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:02.491921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:02.573401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:02.574142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.574502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.574881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.575709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:02.583370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:02.636966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 914/1813] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:02 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:02.640789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:02.656909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a04aa6-6b48-44f0-9e6b-5c980a687e63 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:02.672503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:02.673588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.673928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.674205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.674965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:02.679631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:02.706363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 900/1814] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:02.710707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:02.725093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530ef8-93fd-4198-ab70-ee09fd6b8884 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:02.739544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14391f5a-b152-4844-ad85-6eefb3bb78ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:02.740268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14391f5a-b152-4844-ad85-6eefb3bb78ff admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.740552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14391f5a-b152-4844-ad85-6eefb3bb78ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.740825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14391f5a-b152-4844-ad85-6eefb3bb78ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.741705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14391f5a-b152-4844-ad85-6eefb3bb78ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:02.746637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14391f5a-b152-4844-ad85-6eefb3bb78ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:02.776204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 915/1815] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:02.779568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:02.795690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b365b-2d9b-40a8-b11b-ec8a35dc71b3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:02.810883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:02.811845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.812020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.812549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] PATH_INFO: `/v3/projects/55d5ada1506c4ab4995a9377a6634cb1/users/eaf2af757aee4125a1536ed14b772b95/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.842602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d5ada1506c4ab4995a9377a6634cb1, user_id=eaf2af757aee4125a1536ed14b772b95, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:02.854645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:02.875243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 901/1816] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:02 2022] PUT /identity/v3/projects/55d5ada1506c4ab4995a9377a6634cb1/users/eaf2af757aee4125a1536ed14b772b95/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:02.892076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95613df-26d8-45c1-842c-7b0719f27f30 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:02.892473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95613df-26d8-45c1-842c-7b0719f27f30 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:02.892965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95613df-26d8-45c1-842c-7b0719f27f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:02.978468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b95613df-26d8-45c1-842c-7b0719f27f30 None None] MFA Rules not processed for user `eaf2af757aee4125a1536ed14b772b95`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:02.979978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b95613df-26d8-45c1-842c-7b0719f27f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:03.097785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 916/1817] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:03.098897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:03.272761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b4386-ba00-4c89-8d89-4f6f7903409a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:03.280859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:03.282301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:03.282732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:03.283108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:03.283913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:03.288230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:03.355719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 902/1818] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:03 2022] POST /identity/v3/projects => generated 365 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:03.360476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95613df-26d8-45c1-842c-7b0719f27f30 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:03.394325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95613df-26d8-45c1-842c-7b0719f27f30 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:03.528649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:03.529374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:03.529730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:03.530088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:03.530927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:03.533368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:03.606473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 917/1819] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:03 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:03.607915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:03.623474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477f50c-11a2-4849-b9d9-874e53b6c3ea admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:03.629818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:03.630738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:03.631157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:03.631584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:03.633322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:03.635916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:03.657749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 903/1820] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:03.658668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:03.683355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20783fc-ca7b-4f83-be3e-f1c4359d0b16 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:03.694499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:03.697179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:03.697449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:03.697736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] PATH_INFO: `/v3/projects/3097fa00940043ddbea0805963ce9328/users/436481482e104deaa6e25216f9414ee5/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:03.752702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3097fa00940043ddbea0805963ce9328, user_id=436481482e104deaa6e25216f9414ee5, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:03.755583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:03.784924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 918/1821] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:03 2022] PUT /identity/v3/projects/3097fa00940043ddbea0805963ce9328/users/436481482e104deaa6e25216f9414ee5/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:03.790359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:03.818744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670f82-7bec-4a24-9ee8-0d594031cc34 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:03.977984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:03.992155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2fca7-c568-48a6-9195-edddb6655d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:03.993603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2fca7-c568-48a6-9195-edddb6655d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:03.994054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2fca7-c568-48a6-9195-edddb6655d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:03.994543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2fca7-c568-48a6-9195-edddb6655d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:03.995401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2fca7-c568-48a6-9195-edddb6655d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:03.997358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2fca7-c568-48a6-9195-edddb6655d83 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:04.018213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 904/1822] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:04.022194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:04.036475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efb0e1-00d6-4785-b8c9-c46b601293fb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:04.045240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b655b01-5192-491a-a087-f9c01fda0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:04.046400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b655b01-5192-491a-a087-f9c01fda0a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.046965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b655b01-5192-491a-a087-f9c01fda0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.047481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b655b01-5192-491a-a087-f9c01fda0a01 admin admin] PATH_INFO: `/v3/projects/3097fa00940043ddbea0805963ce9328/users/436481482e104deaa6e25216f9414ee5/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.050830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b655b01-5192-491a-a087-f9c01fda0a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3097fa00940043ddbea0805963ce9328, user_id=436481482e104deaa6e25216f9414ee5, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:04.056066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b655b01-5192-491a-a087-f9c01fda0a01 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:04.062426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bda9af-7371-4205-926c-7165dcdbfc4f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.062963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bda9af-7371-4205-926c-7165dcdbfc4f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.063470 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bda9af-7371-4205-926c-7165dcdbfc4f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.065512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 905/1823] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:12:04.075494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:04.081400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 919/1824] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:04 2022] PUT /identity/v3/projects/3097fa00940043ddbea0805963ce9328/users/436481482e104deaa6e25216f9414ee5/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:04.094314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.094772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.095201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.128418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.129632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.130261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.160929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] MFA Rules not processed for user `436481482e104deaa6e25216f9414ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:04.162184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.167234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] MFA Rules not processed for user `3628a46d8f894d5481112ad00479a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:04.169377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.311194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 920/1825] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:12:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:04.343200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 906/1826] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:04.347133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:04.350696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.368507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:04.371350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.394543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b24245-b4c5-4e61-97e7-ea72fb2687c3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:04.401700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:04.402625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.403058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.403491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.419534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:04.421414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:04.436545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 907/1827] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:04.466684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:04.470036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.470642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b610e9-6096-4cbe-9750-71296c13605e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:04.476943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:04.477928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.478292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.478683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.491664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:04.493349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:04.506294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 921/1828] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:04.522996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:04.525949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.576415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b957655-6ecc-4bb0-85b1-b15d84e13a42 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:04.587330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:04.588863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.589397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.590598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] PATH_INFO: `/v3/projects/8f4e50f1aa0c40a6b2851d5c6045e16d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.594385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f4e50f1aa0c40a6b2851d5c6045e16d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:04.598225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:04.677720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afc7fb-3755-41d6-baab-c6253d47afa4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:04.685252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:04.685925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.688811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.689122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] PATH_INFO: `/v3/users/12998de4e3044171afcb7e5da42edef5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.691108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12998de4e3044171afcb7e5da42edef5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:04.694169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:04.703305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f4e50f1aa0c40a6b2851d5c6045e16d', 'request_id': 'req-db3b6fbe-4a58-427d-a727-05b3ffe56f47'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.715865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f4e50f1aa0c40a6b2851d5c6045e16d', 'request_id': 'req-db3b6fbe-4a58-427d-a727-05b3ffe56f47'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.720230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:04.725474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f4e50f1aa0c40a6b2851d5c6045e16d', 'request_id': 'req-db3b6fbe-4a58-427d-a727-05b3ffe56f47'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.746705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] The token cache is being invalidate because project 8f4e50f1aa0c40a6b2851d5c6045e16d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:04.747558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.765572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db3b6fbe-4a58-427d-a727-05b3ffe56f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.774865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 908/1829] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:04 2022] DELETE /identity/v3/projects/8f4e50f1aa0c40a6b2851d5c6045e16d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:04.777164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:04.787635 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12998de4e3044171afcb7e5da42edef5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.790772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71b9c-e38b-4d97-b460-db1d90916654 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.792669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71b9c-e38b-4d97-b460-db1d90916654 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.793079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71b9c-e38b-4d97-b460-db1d90916654 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.798339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 909/1830] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:12:04 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:12:04.800519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12998de4e3044171afcb7e5da42edef5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.801884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71b9c-e38b-4d97-b460-db1d90916654 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:04.808875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4fb71b9c-e38b-4d97-b460-db1d90916654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.820388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12998de4e3044171afcb7e5da42edef5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.821967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12998de4e3044171afcb7e5da42edef5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.834190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12998de4e3044171afcb7e5da42edef5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:04.853867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 922/1831] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:04 2022] DELETE /identity/v3/users/12998de4e3044171afcb7e5da42edef5 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:04.854666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:04.857403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:04.912366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71b9c-e38b-4d97-b460-db1d90916654 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:04.978420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303b326-4718-4032-bef2-f2ab2ed4942a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:04.987250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:04.989390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:04.990034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:04.990802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] PATH_INFO: `/v3/users/70f6102f3c98410eb195895e1a076d4d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:04.993729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f6102f3c98410eb195895e1a076d4d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:04.997784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:05.031842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:05.033107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:05.033717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:05.034089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:05.037023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:05.092761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f6102f3c98410eb195895e1a076d4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.102413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f6102f3c98410eb195895e1a076d4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.126540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f6102f3c98410eb195895e1a076d4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.127575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f6102f3c98410eb195895e1a076d4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.142812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-85e94fd6-64e1-4a63-96ac-3c21218b0718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f6102f3c98410eb195895e1a076d4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.153632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 923/1832] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:04 2022] DELETE /identity/v3/users/70f6102f3c98410eb195895e1a076d4d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:05.155463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:05.155775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:05.156526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:05.159364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:05.161070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:05.175542 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 910/1833] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:05.176968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:05.178415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:05.181866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:05.183273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:05.198681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd7342-a210-4182-a4b0-c4135cdd9864 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:05.225200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:05.281406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 924/1834] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:12:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:05.332768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:05.334237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:05.338817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:05.339480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:05.340676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:05.341077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] PATH_INFO: `/v3/projects/8a93214740dc4e4483afdab2a92cd9af` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:05.342596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a93214740dc4e4483afdab2a92cd9af)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:05.345148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:05.349534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:05.357355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ab135-2808-4b97-be51-bf5506027918 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:05.392204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:05.392793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:05.393074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:05.393350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:05.432035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:05.436348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:05.453563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a93214740dc4e4483afdab2a92cd9af', 'request_id': 'req-20e35d17-904b-484a-aa22-e9855a01919e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.461123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 925/1835] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:05.469042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a93214740dc4e4483afdab2a92cd9af', 'request_id': 'req-20e35d17-904b-484a-aa22-e9855a01919e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.483088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a93214740dc4e4483afdab2a92cd9af', 'request_id': 'req-20e35d17-904b-484a-aa22-e9855a01919e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.503105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] The token cache is being invalidate because project 8a93214740dc4e4483afdab2a92cd9af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:05.503105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.504420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.510272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 911/1836] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:05 2022] DELETE /identity/v3/projects/8a93214740dc4e4483afdab2a92cd9af => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:05.512238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:05.513512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:05.530823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1cb8b-324e-4e44-b7c5-777310dc6e05 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:05.616126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:05.617754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:05.665127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:05.666154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:05.666325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:05.666607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] PATH_INFO: `/v3/users/6c48c0ac0a9d45cda7e8b710f8c4de01` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:05.667878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c48c0ac0a9d45cda7e8b710f8c4de01)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:05.671943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:05.738037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35d17-904b-484a-aa22-e9855a01919e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:05.744817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:05.745654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:05.746259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:05.746835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] PATH_INFO: `/v3/projects/5176c0cb5fb4400987b1d7c77ceabab6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:05.748182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5176c0cb5fb4400987b1d7c77ceabab6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:05.750977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:05.771702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c48c0ac0a9d45cda7e8b710f8c4de01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.787629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c48c0ac0a9d45cda7e8b710f8c4de01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.807110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c48c0ac0a9d45cda7e8b710f8c4de01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.809667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c48c0ac0a9d45cda7e8b710f8c4de01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.821868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c48c0ac0a9d45cda7e8b710f8c4de01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.832002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 926/1837] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:05 2022] DELETE /identity/v3/users/6c48c0ac0a9d45cda7e8b710f8c4de01 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:05.832976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5176c0cb5fb4400987b1d7c77ceabab6', 'request_id': 'req-eeaa8632-a162-4926-b48c-5e10a0d33d1e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.856356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5176c0cb5fb4400987b1d7c77ceabab6', 'request_id': 'req-eeaa8632-a162-4926-b48c-5e10a0d33d1e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.884143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5176c0cb5fb4400987b1d7c77ceabab6', 'request_id': 'req-eeaa8632-a162-4926-b48c-5e10a0d33d1e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.896723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] The token cache is being invalidate because project 5176c0cb5fb4400987b1d7c77ceabab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:05.897006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.897652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:05.899783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 912/1838] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:05 2022] DELETE /identity/v3/projects/5176c0cb5fb4400987b1d7c77ceabab6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:05.903821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:05.905339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:05.944092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:05.990263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c53865-e105-4782-9523-a5ad05a8a5cb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:05.996178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:05.996681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:05.996964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.006221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] PATH_INFO: `/v3/users/c31c86da27064c7a939dc6a3adc427b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.006221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31c86da27064c7a939dc6a3adc427b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:06.006221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:06.064911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31c86da27064c7a939dc6a3adc427b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.071234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:06.074003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:06.074226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31c86da27064c7a939dc6a3adc427b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.089384 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31c86da27064c7a939dc6a3adc427b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.090055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31c86da27064c7a939dc6a3adc427b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.097314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2afac21b-24d3-42f7-8e54-759175329841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31c86da27064c7a939dc6a3adc427b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.106548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 927/1839] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:05 2022] DELETE /identity/v3/users/c31c86da27064c7a939dc6a3adc427b6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:06.123501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:06.177926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa8632-a162-4926-b48c-5e10a0d33d1e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:06.236994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:06.237745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:06.238090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.238430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] PATH_INFO: `/v3/projects/4cc9f85edece4529b67136e8b50d1422` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.239672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc9f85edece4529b67136e8b50d1422)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:06.241676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:06.293960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc9f85edece4529b67136e8b50d1422', 'request_id': 'req-94e143da-6c53-420d-9c4a-9d970e0e18f4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.301717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc9f85edece4529b67136e8b50d1422', 'request_id': 'req-94e143da-6c53-420d-9c4a-9d970e0e18f4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.311115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc9f85edece4529b67136e8b50d1422', 'request_id': 'req-94e143da-6c53-420d-9c4a-9d970e0e18f4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.325109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] The token cache is being invalidate because project 4cc9f85edece4529b67136e8b50d1422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:06.325474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.326359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-94e143da-6c53-420d-9c4a-9d970e0e18f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.328206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 913/1840] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:06 2022] DELETE /identity/v3/projects/4cc9f85edece4529b67136e8b50d1422 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:06.333029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39f829-63cd-452e-83de-3df5772869bb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:06.333311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39f829-63cd-452e-83de-3df5772869bb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.333590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39f829-63cd-452e-83de-3df5772869bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.355638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da39f829-63cd-452e-83de-3df5772869bb None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:06.359856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da39f829-63cd-452e-83de-3df5772869bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:06.448860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 928/1841] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:06.458217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd22ef-539a-48bf-942d-98c1cf47c0cc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:06.458506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd22ef-539a-48bf-942d-98c1cf47c0cc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.458782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd22ef-539a-48bf-942d-98c1cf47c0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.476069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39f829-63cd-452e-83de-3df5772869bb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:06.478049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da39f829-63cd-452e-83de-3df5772869bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:06.496078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-84bd22ef-539a-48bf-942d-98c1cf47c0cc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:06.498319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-84bd22ef-539a-48bf-942d-98c1cf47c0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:06.510871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 914/1842] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:06.522476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39f829-63cd-452e-83de-3df5772869bb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:06.534730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:06.539213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:06.539416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd22ef-539a-48bf-942d-98c1cf47c0cc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:06.540004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.540420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] PATH_INFO: `/v3/projects/e3e7749c4f0a4e7bb81a3b5278a2ba8d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.542034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e7749c4f0a4e7bb81a3b5278a2ba8d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:06.546802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:06.577400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd22ef-539a-48bf-942d-98c1cf47c0cc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:06.595823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:06.596505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:06.596785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.597043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.597708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:06.598012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:06.599636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:06.642604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 915/1843] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:06.643632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:06.664397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d66c-db42-4cae-9853-9e5dbc771b1f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:06.672970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:06.681372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e7749c4f0a4e7bb81a3b5278a2ba8d', 'request_id': 'req-aff61520-b009-45b6-a762-9bcfa3a62831'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.694619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e7749c4f0a4e7bb81a3b5278a2ba8d', 'request_id': 'req-aff61520-b009-45b6-a762-9bcfa3a62831'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.713993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e7749c4f0a4e7bb81a3b5278a2ba8d', 'request_id': 'req-aff61520-b009-45b6-a762-9bcfa3a62831'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.765482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] The token cache is being invalidate because project e3e7749c4f0a4e7bb81a3b5278a2ba8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:06.765777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.768333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:06.769668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:06.777220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 929/1844] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:06 2022] DELETE /identity/v3/projects/e3e7749c4f0a4e7bb81a3b5278a2ba8d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:06.780947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:06.786683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:06.814827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f27107-e654-47b8-9529-bd146da05d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:06.815708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27107-e654-47b8-9529-bd146da05d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:06.816281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27107-e654-47b8-9529-bd146da05d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.816696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27107-e654-47b8-9529-bd146da05d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.817500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f27107-e654-47b8-9529-bd146da05d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:06.822648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f27107-e654-47b8-9529-bd146da05d14 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:06.890369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 916/1845] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:06 2022] POST /identity/v3/projects => generated 371 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:06.890606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:06.891012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:06.893687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:06.905340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff61520-b009-45b6-a762-9bcfa3a62831 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:06.935425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:06.938906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:07.068669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.069756 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.070269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.071102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] PATH_INFO: `/v3/projects/39f5e1c8e8604bdf9763ac9094698b59` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.073435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f5e1c8e8604bdf9763ac9094698b59)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.077081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.082824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 917/1846] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:07.092327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.099991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:07.132680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae60bb-bc8c-4bde-81f4-fb497463f465 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.149145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac3b33-a069-4906-b99d-a1e17d2a5995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.151524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac3b33-a069-4906-b99d-a1e17d2a5995 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.154301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac3b33-a069-4906-b99d-a1e17d2a5995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.155480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac3b33-a069-4906-b99d-a1e17d2a5995 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.156584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac3b33-a069-4906-b99d-a1e17d2a5995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.163201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac3b33-a069-4906-b99d-a1e17d2a5995 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.206551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 918/1847] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:07.209900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a168aba-6140-4479-af39-514fa8c07834 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.210196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a168aba-6140-4479-af39-514fa8c07834 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.212330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a168aba-6140-4479-af39-514fa8c07834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.248793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f5e1c8e8604bdf9763ac9094698b59', 'request_id': 'req-612b5206-6467-46f5-8996-821e12fac7ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:07.256980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6a168aba-6140-4479-af39-514fa8c07834 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:07.257140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f5e1c8e8604bdf9763ac9094698b59', 'request_id': 'req-612b5206-6467-46f5-8996-821e12fac7ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:07.258666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6a168aba-6140-4479-af39-514fa8c07834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:07.281821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f5e1c8e8604bdf9763ac9094698b59', 'request_id': 'req-612b5206-6467-46f5-8996-821e12fac7ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:07.322565 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] The token cache is being invalidate because project 39f5e1c8e8604bdf9763ac9094698b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:07.325777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:07.327672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:07.337325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 930/1848] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:06 2022] DELETE /identity/v3/projects/39f5e1c8e8604bdf9763ac9094698b59 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:07.338796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.348050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:07.382615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:07.386885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 919/1849] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:07.389950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed67ce-bdaa-4241-b421-e42428530022 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.390297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed67ce-bdaa-4241-b421-e42428530022 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.390580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed67ce-bdaa-4241-b421-e42428530022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.458964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-10ed67ce-bdaa-4241-b421-e42428530022 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:07.460954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-10ed67ce-bdaa-4241-b421-e42428530022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:07.471573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 920/1850] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:07.473083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed67ce-bdaa-4241-b421-e42428530022 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.488996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed67ce-bdaa-4241-b421-e42428530022 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.499087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aed31b-1e9b-477e-a620-051b76b59cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.500188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aed31b-1e9b-477e-a620-051b76b59cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.500632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aed31b-1e9b-477e-a620-051b76b59cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.501026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aed31b-1e9b-477e-a620-051b76b59cc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.501750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aed31b-1e9b-477e-a620-051b76b59cc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:07.502105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aed31b-1e9b-477e-a620-051b76b59cc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.508588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aed31b-1e9b-477e-a620-051b76b59cc1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.522667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b5206-6467-46f5-8996-821e12fac7ff admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.524197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 921/1851] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:07.525685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430911-b050-4b2e-935b-8228f3167117 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.526027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430911-b050-4b2e-935b-8228f3167117 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.526366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430911-b050-4b2e-935b-8228f3167117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.530793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.531415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.531771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.531971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.532732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.534392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.552257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-41430911-b050-4b2e-935b-8228f3167117 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:07.553888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-41430911-b050-4b2e-935b-8228f3167117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:07.558861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 931/1852] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:07.559788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.568193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 922/1853] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:07.569583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430911-b050-4b2e-935b-8228f3167117 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.582623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430911-b050-4b2e-935b-8228f3167117 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.585471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1206e95-13aa-429d-beec-0cd4ee90fac7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.588864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.589472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.589751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.590025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.590687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.592827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.592952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.593460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.593725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.594069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.594703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.597248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.611005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 923/1854] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:07.612999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.626827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c2e5-9f70-4c80-8ba2-a8a1a35fe58c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.632242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.633083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.633656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.634085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.635014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:07.635532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.637759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.647801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:07.662115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 932/1855] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:07.662306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 924/1856] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:07.662383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.665046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.682207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f62a4-6ca7-4dc1-b984-4190d454868a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.815123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.816083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.816610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.817076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.817851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.820084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.836794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a2bd6-f8ec-4bd7-91da-7c19b720c4a9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.858275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c0fcf-8f20-4053-a3ea-86a7ed154b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.858969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c0fcf-8f20-4053-a3ea-86a7ed154b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.862024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c0fcf-8f20-4053-a3ea-86a7ed154b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.862243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 925/1857] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:07.862320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c0fcf-8f20-4053-a3ea-86a7ed154b06 admin admin] PATH_INFO: `/v3/projects/712b075e2e904cb384fa8dc7e261de81/users/bdc6fc3a678e4b179631a3df57f13f70/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.862440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.891062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83de2f-4c78-45c3-ab60-97a8753c65f1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:07.895794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c0fcf-8f20-4053-a3ea-86a7ed154b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712b075e2e904cb384fa8dc7e261de81, user_id=bdc6fc3a678e4b179631a3df57f13f70, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.896616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:07.897347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.897732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.898108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:07.898523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c0fcf-8f20-4053-a3ea-86a7ed154b06 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.899611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:07.901761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:07.936615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 926/1858] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:07.937369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 933/1859] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:07 2022] PUT /identity/v3/projects/712b075e2e904cb384fa8dc7e261de81/users/bdc6fc3a678e4b179631a3df57f13f70/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:07.937487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:07.947648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c93a-b5ee-4326-9022-2ee1f03bc3b5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:07.948409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c93a-b5ee-4326-9022-2ee1f03bc3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:07.949329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c93a-b5ee-4326-9022-2ee1f03bc3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.004689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-79f7c93a-b5ee-4326-9022-2ee1f03bc3b5 None None] MFA Rules not processed for user `bdc6fc3a678e4b179631a3df57f13f70`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:08.007322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79f7c93a-b5ee-4326-9022-2ee1f03bc3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:08.014509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e248d-a233-4c79-95dd-c7109e535ee0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.021785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.022836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.023120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.023525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.024845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.026629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.041605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 927/1860] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:08.042596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.054455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1404046-630a-436b-b28a-85fc9e9cb369 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.062259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.063035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.063426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.063973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.064787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.067142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.093514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 928/1861] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:08 2022] POST /identity/v3/users => generated 283 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:08.094604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.098847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:08.105073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcbff6-ae65-446c-bb1e-575271c5b9ea admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.109772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.110381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.110662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.111155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.111848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 934/1862] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:08.111946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.112294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c93a-b5ee-4326-9022-2ee1f03bc3b5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.113224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.125924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c93a-b5ee-4326-9022-2ee1f03bc3b5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.130725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 929/1863] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:08.130725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.130725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:08.131058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.131935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.132233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.132496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.133182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.134767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.148502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 935/1864] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:08.149875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.160469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d076bcc-aae1-475a-b498-07755e24949d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.165422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.166110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.166395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.166673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] PATH_INFO: `/v3/projects/918b2c56742d422abd1c168fc3e63610/users/1e07c23971e743c3b568f4aa4a967958/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.174330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:08.185353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918b2c56742d422abd1c168fc3e63610, user_id=1e07c23971e743c3b568f4aa4a967958, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.188201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.199266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db70af-d256-4758-ba9c-7f19f0113015 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.202778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 936/1865] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:08 2022] PUT /identity/v3/projects/918b2c56742d422abd1c168fc3e63610/users/1e07c23971e743c3b568f4aa4a967958/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:08.203347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.250424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.250956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.251241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.251508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.266512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e6780-b26b-4a86-9395-5fa2ee41da65 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.271925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71f108-cbaf-49b1-9837-279cce09d0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.272314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71f108-cbaf-49b1-9837-279cce09d0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.272585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71f108-cbaf-49b1-9837-279cce09d0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.272858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71f108-cbaf-49b1-9837-279cce09d0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.273610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71f108-cbaf-49b1-9837-279cce09d0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.275456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71f108-cbaf-49b1-9837-279cce09d0e7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.292730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 937/1866] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:08.294672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53b8f5-fd4e-4a29-a31e-0e25926701bc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.294963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53b8f5-fd4e-4a29-a31e-0e25926701bc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.295241 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53b8f5-fd4e-4a29-a31e-0e25926701bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.316818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.318423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.326931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1b53b8f5-fd4e-4a29-a31e-0e25926701bc None None] MFA Rules not processed for user `1e07c23971e743c3b568f4aa4a967958`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:08.327924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 930/1867] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:08.328558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1b53b8f5-fd4e-4a29-a31e-0e25926701bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:08.328855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.339777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571c0f0-ab4a-45e2-8670-08c42ef44f23 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.350708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.351511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.352253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.352500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] PATH_INFO: `/v3/projects/e6e75a99172649c6a87194f93c2f408c/users/abd0c171b0d84955bacf3996b0fd645e/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.373494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e75a99172649c6a87194f93c2f408c, user_id=abd0c171b0d84955bacf3996b0fd645e, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.376674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.392787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 931/1868] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:08 2022] PUT /identity/v3/projects/e6e75a99172649c6a87194f93c2f408c/users/abd0c171b0d84955bacf3996b0fd645e/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:08.393553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.447129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 938/1869] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:08.448821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0796-74ab-4062-bc44-aa7fcc6d572a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.449309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0796-74ab-4062-bc44-aa7fcc6d572a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.449785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0796-74ab-4062-bc44-aa7fcc6d572a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.538424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-adfb0796-74ab-4062-bc44-aa7fcc6d572a None None] MFA Rules not processed for user `abd0c171b0d84955bacf3996b0fd645e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:08.541854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-adfb0796-74ab-4062-bc44-aa7fcc6d572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:08.554247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daba74-4f05-4c64-9665-5ace70c3b43d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.567376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.568065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.568349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.568624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] PATH_INFO: `/v3/users/bdc6fc3a678e4b179631a3df57f13f70` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.570662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc6fc3a678e4b179631a3df57f13f70)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.595738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.650649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:08.651742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:08.698312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 939/1870] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:08.698312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0796-74ab-4062-bc44-aa7fcc6d572a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.704759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc6fc3a678e4b179631a3df57f13f70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:08.706460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc6fc3a678e4b179631a3df57f13f70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:08.722799 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc6fc3a678e4b179631a3df57f13f70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:08.725542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0796-74ab-4062-bc44-aa7fcc6d572a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:08.738761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35467f83-69dd-4834-aa40-f35cee3205aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:08.741730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35467f83-69dd-4834-aa40-f35cee3205aa admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.743143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35467f83-69dd-4834-aa40-f35cee3205aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.743824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35467f83-69dd-4834-aa40-f35cee3205aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.745396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35467f83-69dd-4834-aa40-f35cee3205aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:08.747025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc6fc3a678e4b179631a3df57f13f70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:08.749078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc6fc3a678e4b179631a3df57f13f70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:08.752526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35467f83-69dd-4834-aa40-f35cee3205aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:08.767865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 932/1871] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:08 2022] DELETE /identity/v3/users/bdc6fc3a678e4b179631a3df57f13f70 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:08.772759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:08.774729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:08.837953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 940/1872] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:08 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:08.840661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:08.840661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:08.840889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:08.874741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:08.875880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:08.937706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005141e-0c8b-4cc1-900f-486b7ab8b3a1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.038425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.039051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.039381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.039661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.041145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 941/1873] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:09.042427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.044323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.046869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.049809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:09.091500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7100c9-1269-4671-820e-aafd16e43f5a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.096896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 933/1874] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:08 2022] POST /identity/v3/projects => generated 367 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:09.101898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.174194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:09.265625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281d3b-35a5-4bcc-9a04-466fd538ca83 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.289554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.290708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.290998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.291274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.292340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.292491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.293322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.294668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.295169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.295654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.296561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.299004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.328164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 934/1875] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:09 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:09.329972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 942/1876] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:09 2022] POST /identity/v3/users => generated 290 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:09.330836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.334171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:09.335459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.353336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100042a4-1630-4018-8aba-5916cac2a2d5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.368502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.369206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.369558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.369898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.370611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.373924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.395162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 943/1877] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:09.398636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.423588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbbcd2-0dff-4173-9acf-2eaf313953f1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.429100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6888797-9343-4da6-958b-e8353ad27f1c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.433976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.435198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.436001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.437224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.440402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.441044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.441326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.441599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.442256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.443991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.450228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.453114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.470791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 944/1878] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:09.470791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.479533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 935/1879] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:09.480480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.490090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd2774-c9a0-4f30-b886-48379eeee5bc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.502055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.502822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.503009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.503291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] PATH_INFO: `/v3/projects/8bca5ba1e0674f0393cc12dfc6fd13c7/users/0f74691aa45240969576948e8a38bc50/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.503662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670c683-50df-418c-90b3-39a5ab2d247f service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.514431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.515119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.515400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.515694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] PATH_INFO: `/v3/projects/764781db6e4e45d4b71bb00c131a9533/users/abf3cee12c8b48dc8597ad95fe512001/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.548761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bca5ba1e0674f0393cc12dfc6fd13c7, user_id=0f74691aa45240969576948e8a38bc50, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.551344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.559690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] RBAC: Authorizing `identity:create_grant(project_id=764781db6e4e45d4b71bb00c131a9533, user_id=abf3cee12c8b48dc8597ad95fe512001, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.563712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.574360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 945/1880] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:09 2022] PUT /identity/v3/projects/8bca5ba1e0674f0393cc12dfc6fd13c7/users/0f74691aa45240969576948e8a38bc50/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:09.578231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.594317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:09.595140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 936/1881] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:09 2022] PUT /identity/v3/projects/764781db6e4e45d4b71bb00c131a9533/users/abf3cee12c8b48dc8597ad95fe512001/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:09.599343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.678192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd783904-c974-47e8-82c5-0d93c1efeced admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.686997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.687618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.687928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.688201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.688847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.689455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e3830-d03e-46bc-a694-c92501e38cd8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.690618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.694433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.695105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.696245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.696808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.697553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.699159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.703470 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 937/1882] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:09.707398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.711400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 946/1883] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:09.715371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.717925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae427fb-3047-49d0-993e-a485cee2ac90 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.723506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9e47f-a58b-480d-9232-34e5de067efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.723933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e47f-a58b-480d-9232-34e5de067efc admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.724217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e47f-a58b-480d-9232-34e5de067efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.724493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e47f-a58b-480d-9232-34e5de067efc admin admin] PATH_INFO: `/v3/projects/764781db6e4e45d4b71bb00c131a9533/users/abf3cee12c8b48dc8597ad95fe512001/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.726393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9e47f-a58b-480d-9232-34e5de067efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=764781db6e4e45d4b71bb00c131a9533, user_id=abf3cee12c8b48dc8597ad95fe512001, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.727998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b30944-59d3-4dd6-ac04-913fbfe056f6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.729610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9e47f-a58b-480d-9232-34e5de067efc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.733157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a09a43-78d1-4460-b6c3-0fa273bdcc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.734755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09a43-78d1-4460-b6c3-0fa273bdcc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.735134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09a43-78d1-4460-b6c3-0fa273bdcc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.735540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09a43-78d1-4460-b6c3-0fa273bdcc09 admin admin] PATH_INFO: `/v3/projects/8bca5ba1e0674f0393cc12dfc6fd13c7/users/0f74691aa45240969576948e8a38bc50/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.737999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a09a43-78d1-4460-b6c3-0fa273bdcc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bca5ba1e0674f0393cc12dfc6fd13c7, user_id=0f74691aa45240969576948e8a38bc50, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.740823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a09a43-78d1-4460-b6c3-0fa273bdcc09 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.747850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 938/1884] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:09 2022] PUT /identity/v3/projects/764781db6e4e45d4b71bb00c131a9533/users/abf3cee12c8b48dc8597ad95fe512001/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:09.753087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.753388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.753666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.755029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 947/1885] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:09 2022] PUT /identity/v3/projects/8bca5ba1e0674f0393cc12dfc6fd13c7/users/0f74691aa45240969576948e8a38bc50/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:09.767092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.767406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.767667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.793871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] MFA Rules not processed for user `abf3cee12c8b48dc8597ad95fe512001`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:09.795290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:09.803592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] MFA Rules not processed for user `0f74691aa45240969576948e8a38bc50`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:09.805598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:09.808436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:09.885166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 939/1886] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:09.886236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.892903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:09.893342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 948/1887] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:09.897779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.899385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:09.966654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b137dd8-e32b-4699-9f9d-56e91cbf0d9c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.970404 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.971087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.971430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.971783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.971900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069f1ad-8e1b-49bb-9b01-abb61f2ce21e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:09.977419 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:09.978704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:09.978987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:09.979286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] PATH_INFO: `/v3/projects/712b075e2e904cb384fa8dc7e261de81` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:09.980413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=712b075e2e904cb384fa8dc7e261de81)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.982125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:09.982471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.983674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:09.994423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 949/1888] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:09.995323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:09.996720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:10.010222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8fc2f-0295-49a5-bf00-1ac8f2c0b615 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.015837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b28215-a4b0-4d1e-95b4-4b31ee459b5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.016502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b28215-a4b0-4d1e-95b4-4b31ee459b5b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.016777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b28215-a4b0-4d1e-95b4-4b31ee459b5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.017116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b28215-a4b0-4d1e-95b4-4b31ee459b5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.026760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b28215-a4b0-4d1e-95b4-4b31ee459b5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.028216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b28215-a4b0-4d1e-95b4-4b31ee459b5b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.033247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:10.045385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 950/1889] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:10.061664 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712b075e2e904cb384fa8dc7e261de81', 'request_id': 'req-b31ca50a-91dc-4b80-a687-d07ba940a896'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:10.067849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712b075e2e904cb384fa8dc7e261de81', 'request_id': 'req-b31ca50a-91dc-4b80-a687-d07ba940a896'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:10.076723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712b075e2e904cb384fa8dc7e261de81', 'request_id': 'req-b31ca50a-91dc-4b80-a687-d07ba940a896'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:10.090438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] The token cache is being invalidate because project 712b075e2e904cb384fa8dc7e261de81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:10.090729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:10.091603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:10.093830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 940/1890] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:09 2022] DELETE /identity/v3/projects/712b075e2e904cb384fa8dc7e261de81 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:10.098512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8cf57-6c0c-4903-94a9-933170aefbc8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.098861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8cf57-6c0c-4903-94a9-933170aefbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.099184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8cf57-6c0c-4903-94a9-933170aefbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.100742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.103002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:10.120097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-12e8cf57-6c0c-4903-94a9-933170aefbc8 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:10.121280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12e8cf57-6c0c-4903-94a9-933170aefbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:10.156401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:10.181383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ca50a-91dc-4b80-a687-d07ba940a896 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.185207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.185842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.186162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.186437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.188721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:10.191992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 951/1891] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:10.199616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d543bd-382b-4180-b621-f4b456419dcf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.200295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d543bd-382b-4180-b621-f4b456419dcf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.200295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d543bd-382b-4180-b621-f4b456419dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.224770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-99d543bd-382b-4180-b621-f4b456419dcf None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:10.225991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99d543bd-382b-4180-b621-f4b456419dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:10.234863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 952/1892] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:10.238881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d543bd-382b-4180-b621-f4b456419dcf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.249027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d543bd-382b-4180-b621-f4b456419dcf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.254381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.255371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.255371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.255556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.256184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:10.256493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.257120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.258112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.259135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.274282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 941/1893] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:10.278616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 953/1894] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:12:10.283180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.298040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66338780-0bf2-440a-9d26-7ff463fe27b2 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.305727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.306498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.306784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.307329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.307934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.310373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.357548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 942/1895] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:10 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:10.358507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.375302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d33830-4d42-44bb-bf11-7d02d7dca1c7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.453786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.454402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.454682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.454951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.455589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.457552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.486650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 954/1896] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:10 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:10.492496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.505033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd6f5-d15d-47a3-badc-61981fe3aeb1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.512321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.513014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.513382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.513840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.514966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.516753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.535540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 943/1897] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:10.540199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.554023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd1e36-cb85-40ae-8c17-8cbad7dccbd8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.561659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682da27c-7209-4e15-a564-0487297f8912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.563299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682da27c-7209-4e15-a564-0487297f8912 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.563871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682da27c-7209-4e15-a564-0487297f8912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.564243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682da27c-7209-4e15-a564-0487297f8912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.565018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682da27c-7209-4e15-a564-0487297f8912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.568298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682da27c-7209-4e15-a564-0487297f8912 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.589751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 955/1898] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:10.598553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.616023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d378-5733-4c7f-bfcf-92a659e3faec admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.635270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.636138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.636413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.637422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] PATH_INFO: `/v3/projects/d742327ee5064515ac02daa3671cf777/users/ec13ab2b6ecd483cac150babbfa6c6b5/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.664702 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d742327ee5064515ac02daa3671cf777, user_id=ec13ab2b6ecd483cac150babbfa6c6b5, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.667519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:10.683233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 944/1899] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:10 2022] PUT /identity/v3/projects/d742327ee5064515ac02daa3671cf777/users/ec13ab2b6ecd483cac150babbfa6c6b5/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:10.694612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.694826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.694999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.767597 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] MFA Rules not processed for user `ec13ab2b6ecd483cac150babbfa6c6b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:10.769749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:10.872130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:10.892167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 956/1900] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:10.904280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:10.978939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ad84d-e1cf-49c4-9571-62136e2a9b1d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:10.982752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f2323-c9ab-4faa-860a-081cc0a89db3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:10.983429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f2323-c9ab-4faa-860a-081cc0a89db3 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:10.983798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f2323-c9ab-4faa-860a-081cc0a89db3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:10.984144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f2323-c9ab-4faa-860a-081cc0a89db3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:10.993510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f2323-c9ab-4faa-860a-081cc0a89db3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:10.994999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f2323-c9ab-4faa-860a-081cc0a89db3 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:11.004157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 945/1901] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:11.747369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:11.748946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:11.822079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00316f-20e9-4f7c-ba73-85ea4b7ee94f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:11.827450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:11.828530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:11.828817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:11.829099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] PATH_INFO: `/v3/users/1e07c23971e743c3b568f4aa4a967958` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:11.830236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e07c23971e743c3b568f4aa4a967958)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:11.832186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:11.885440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e07c23971e743c3b568f4aa4a967958'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:11.892522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e07c23971e743c3b568f4aa4a967958'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:11.905821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e07c23971e743c3b568f4aa4a967958'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:11.906527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e07c23971e743c3b568f4aa4a967958'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:11.913388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e07c23971e743c3b568f4aa4a967958'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:11.920233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 957/1902] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:11 2022] DELETE /identity/v3/users/1e07c23971e743c3b568f4aa4a967958 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:11.924839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:11.925132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:11.925409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:11.944898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:11.946646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:11.983765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:12.014610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 946/1903] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:12.027207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:12.028816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:12.089639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227ff27-fe08-4728-b268-fbb72bd590a3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:12.093369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcfd87-9ac8-414f-9c4b-f9df434296db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:12.094057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcfd87-9ac8-414f-9c4b-f9df434296db service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:12.094399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcfd87-9ac8-414f-9c4b-f9df434296db service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:12.094732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcfd87-9ac8-414f-9c4b-f9df434296db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:12.103805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcfd87-9ac8-414f-9c4b-f9df434296db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:12.105292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcfd87-9ac8-414f-9c4b-f9df434296db service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:12.114797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 958/1904] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:12.694854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:12.697236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:12.731259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae0833-1ae7-4ded-9cb1-bf5a86aad58b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:12.755466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:12.756880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:12.757264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:12.757633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] PATH_INFO: `/v3/projects/918b2c56742d422abd1c168fc3e63610` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:12.759163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] RBAC: Authorizing `identity:delete_project(project_id=918b2c56742d422abd1c168fc3e63610)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:12.761964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:12.835225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918b2c56742d422abd1c168fc3e63610', 'request_id': 'req-b537bbc3-bc28-4538-95d3-a2338772681f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:12.855792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918b2c56742d422abd1c168fc3e63610', 'request_id': 'req-b537bbc3-bc28-4538-95d3-a2338772681f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:12.863022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918b2c56742d422abd1c168fc3e63610', 'request_id': 'req-b537bbc3-bc28-4538-95d3-a2338772681f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:12.883881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d48b7-d79a-4373-bf80-6977f044968e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:12.884819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d48b7-d79a-4373-bf80-6977f044968e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:12.885413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d48b7-d79a-4373-bf80-6977f044968e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:12.887730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 959/1905] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:12:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:12:12.891060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] The token cache is being invalidate because project 918b2c56742d422abd1c168fc3e63610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:12.892326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:12.894016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b537bbc3-bc28-4538-95d3-a2338772681f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:12.899688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 947/1906] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:12 2022] DELETE /identity/v3/projects/918b2c56742d422abd1c168fc3e63610 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:12.904392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d48b7-d79a-4373-bf80-6977f044968e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:12.906264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-af4d48b7-d79a-4373-bf80-6977f044968e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:12.930887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:12.931294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:12.931583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:12.933656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:12.979935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:12.983733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:13.096066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d48b7-d79a-4373-bf80-6977f044968e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:13.114592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:13.115257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:13.115534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:13.117004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] PATH_INFO: `/v3/users/0f74691aa45240969576948e8a38bc50` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:13.118927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f74691aa45240969576948e8a38bc50)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:13.121448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:13.176416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 948/1907] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:12:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:13.235835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f74691aa45240969576948e8a38bc50'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:13.251943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f74691aa45240969576948e8a38bc50'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:13.276642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f74691aa45240969576948e8a38bc50'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:13.277698 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f74691aa45240969576948e8a38bc50'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:13.289925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f74691aa45240969576948e8a38bc50'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:13.297248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 960/1908] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:12 2022] DELETE /identity/v3/users/0f74691aa45240969576948e8a38bc50 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:13.383270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:13.386081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:13.402306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd481-6f13-4847-95f6-ac849533a1a7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:13.521080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:13.558368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243b925-a703-420d-86c3-edb3194a97fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:13.558368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243b925-a703-420d-86c3-edb3194a97fd service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:13.558628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243b925-a703-420d-86c3-edb3194a97fd service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:13.559622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243b925-a703-420d-86c3-edb3194a97fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:13.561079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5243b925-a703-420d-86c3-edb3194a97fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:13.587718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243b925-a703-420d-86c3-edb3194a97fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:13.589430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243b925-a703-420d-86c3-edb3194a97fd service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:13.620864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 949/1909] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:13.935634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:13.937106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:14.002179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e9850-e240-4b36-8c5d-59f5047a9251 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.006762 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.007391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.007671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.008598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] PATH_INFO: `/v3/projects/8bca5ba1e0674f0393cc12dfc6fd13c7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.009707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bca5ba1e0674f0393cc12dfc6fd13c7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.011600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.046190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:14.062347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bca5ba1e0674f0393cc12dfc6fd13c7', 'request_id': 'req-4217aed1-7d4a-4301-96ed-d4dd279fd24b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:14.067976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bca5ba1e0674f0393cc12dfc6fd13c7', 'request_id': 'req-4217aed1-7d4a-4301-96ed-d4dd279fd24b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:14.075803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bca5ba1e0674f0393cc12dfc6fd13c7', 'request_id': 'req-4217aed1-7d4a-4301-96ed-d4dd279fd24b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:14.086672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] The token cache is being invalidate because project 8bca5ba1e0674f0393cc12dfc6fd13c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:14.086962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:14.087585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4217aed1-7d4a-4301-96ed-d4dd279fd24b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:14.089098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 961/1910] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:13 2022] DELETE /identity/v3/projects/8bca5ba1e0674f0393cc12dfc6fd13c7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:14.094038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797234f1-2275-4ebe-ac15-fa672c1f1d43 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.094498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797234f1-2275-4ebe-ac15-fa672c1f1d43 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.094594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797234f1-2275-4ebe-ac15-fa672c1f1d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.113611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-797234f1-2275-4ebe-ac15-fa672c1f1d43 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:14.114800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-797234f1-2275-4ebe-ac15-fa672c1f1d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:14.172320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 950/1911] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:14.181670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81284fca-ad7d-426f-b4dc-2769ac97f8d3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.181920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81284fca-ad7d-426f-b4dc-2769ac97f8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.182190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81284fca-ad7d-426f-b4dc-2769ac97f8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.202856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-81284fca-ad7d-426f-b4dc-2769ac97f8d3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:14.204074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-81284fca-ad7d-426f-b4dc-2769ac97f8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:14.212110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 962/1912] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:14.216267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797234f1-2275-4ebe-ac15-fa672c1f1d43 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:14.225708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797234f1-2275-4ebe-ac15-fa672c1f1d43 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.230660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.231248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.231525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.231823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.232467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:14.232765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.234333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.246319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 951/1913] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:14.250462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81284fca-ad7d-426f-b4dc-2769ac97f8d3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:14.259456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81284fca-ad7d-426f-b4dc-2769ac97f8d3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.264171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.264761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.265032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.265320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.265944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.267871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.294539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 963/1914] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:14 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:14.298537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:14.308933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5154dd-8f63-4d22-90ae-fe3dbfdf76b9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.363286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.364271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.364564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.364840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.365474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.368041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.388186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 952/1915] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:14 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:14.392322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:14.402486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26bbb7-7627-4b26-bb70-810cb025fd1b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.407171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.407794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.408079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.408353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.408989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.410588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.421645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 964/1916] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:14.425916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:14.437084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53961fd7-f1a7-47c9-80a5-8a4f9619630c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.441729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb492652-64d8-447f-85bd-c929d31a7939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.442331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb492652-64d8-447f-85bd-c929d31a7939 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.442604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb492652-64d8-447f-85bd-c929d31a7939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.442869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb492652-64d8-447f-85bd-c929d31a7939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.443497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb492652-64d8-447f-85bd-c929d31a7939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.445214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb492652-64d8-447f-85bd-c929d31a7939 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.459070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 953/1917] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:14.463235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:14.473520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16b4d4-10e8-4e1d-8b2f-963361f1e9b4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.478905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.479570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.479876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.480146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] PATH_INFO: `/v3/projects/079028929bb4478c8c3c2b474e74f109/users/7bb9cf9d423f4fc3b2805bba0db064ee/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.497767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=079028929bb4478c8c3c2b474e74f109, user_id=7bb9cf9d423f4fc3b2805bba0db064ee, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.500627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.524040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 965/1918] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:14 2022] PUT /identity/v3/projects/079028929bb4478c8c3c2b474e74f109/users/7bb9cf9d423f4fc3b2805bba0db064ee/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:14.529077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4347c14-347a-40b2-902d-ea434d2aca6b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.529293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4347c14-347a-40b2-902d-ea434d2aca6b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.529738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4347c14-347a-40b2-902d-ea434d2aca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.565153 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f4347c14-347a-40b2-902d-ea434d2aca6b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:14.574591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f4347c14-347a-40b2-902d-ea434d2aca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:14.651090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:14.678319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 954/1919] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:14.688822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:14.691155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:14.758186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d0bc-b4be-4b41-8a95-a94a029fdb9c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:14.762796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:14.763416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:14.763706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:14.764028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:14.776018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:14.777643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:14.788525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 966/1920] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:15.711662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff901ef-91ce-4723-b280-feafaf29cf00 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:15.712161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff901ef-91ce-4723-b280-feafaf29cf00 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:15.712257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff901ef-91ce-4723-b280-feafaf29cf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:15.788844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8ff901ef-91ce-4723-b280-feafaf29cf00 None None] MFA Rules not processed for user `7bb9cf9d423f4fc3b2805bba0db064ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:15.790877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8ff901ef-91ce-4723-b280-feafaf29cf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:15.944523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 955/1921] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:15.949349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:15.967607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062237f-76b9-4cc9-aa16-c29bfaed8eb6 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:15.973225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:15.973702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:15.973980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:15.974979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:15.975622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:15.978166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:16.013340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 967/1922] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:15 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:16.017550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff901ef-91ce-4723-b280-feafaf29cf00 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:16.034459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff901ef-91ce-4723-b280-feafaf29cf00 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:16.123348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:16.124079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:16.124421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:16.124703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:16.125364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:16.127366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:16.155962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 956/1923] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:16 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:16.160045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:16.197623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a90e0-ba99-4ccc-9c8a-07c79fed8571 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:16.217324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:16.218328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:16.218880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:16.219247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:16.220057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:16.226041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:16.268072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:16.279476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 968/1924] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:16.286464 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c766-3210-4f35-88a3-c0eab6e3f9ed admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:16.300312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:16.301011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:16.301275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:16.301958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] PATH_INFO: `/v3/projects/fcaa9492fa2849e1bc7ef707cb4d0219/users/9d67b1e32ef34deba8791de44bb35708/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:16.320134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcaa9492fa2849e1bc7ef707cb4d0219, user_id=9d67b1e32ef34deba8791de44bb35708, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:16.323171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:16.355445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 957/1925] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:16 2022] PUT /identity/v3/projects/fcaa9492fa2849e1bc7ef707cb4d0219/users/9d67b1e32ef34deba8791de44bb35708/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:16.357416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:16.370288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376861c9-adfb-4500-b0a0-4e9ae50d7412 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:16.438346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:16.524473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:16.525120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:16.525396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:16.525667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:16.526313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:16.532677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:16.546539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 969/1926] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:16.553759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:16.571128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afa1a-dc17-4da6-a6ae-b41250bdfd3b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:16.581733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a674d-7e49-454b-80e7-79272cb38f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:16.582732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a674d-7e49-454b-80e7-79272cb38f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:16.583340 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a674d-7e49-454b-80e7-79272cb38f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:16.583847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a674d-7e49-454b-80e7-79272cb38f23 admin admin] PATH_INFO: `/v3/projects/fcaa9492fa2849e1bc7ef707cb4d0219/users/9d67b1e32ef34deba8791de44bb35708/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:16.587667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a674d-7e49-454b-80e7-79272cb38f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcaa9492fa2849e1bc7ef707cb4d0219, user_id=9d67b1e32ef34deba8791de44bb35708, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:16.591393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a674d-7e49-454b-80e7-79272cb38f23 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:16.606352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 958/1927] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:16 2022] PUT /identity/v3/projects/fcaa9492fa2849e1bc7ef707cb4d0219/users/9d67b1e32ef34deba8791de44bb35708/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:17.086191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:17.088387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:17.109652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f0440-d282-4de5-b780-2c1b2453d4d7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:17.225744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:17.227102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:17.227436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:17.227713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] PATH_INFO: `/v3/users/abd0c171b0d84955bacf3996b0fd645e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:17.229021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd0c171b0d84955bacf3996b0fd645e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:17.231038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:17.301364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd0c171b0d84955bacf3996b0fd645e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:17.312230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd0c171b0d84955bacf3996b0fd645e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:17.330676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd0c171b0d84955bacf3996b0fd645e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:17.331463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd0c171b0d84955bacf3996b0fd645e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:17.344604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd0c171b0d84955bacf3996b0fd645e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:17.358215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 970/1928] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:17 2022] DELETE /identity/v3/users/abd0c171b0d84955bacf3996b0fd645e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:17.362777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5f5a8-d144-4914-b93b-7244d3f7f463 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:17.363248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5f5a8-d144-4914-b93b-7244d3f7f463 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:17.363335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5f5a8-d144-4914-b93b-7244d3f7f463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:17.384328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-55a5f5a8-d144-4914-b93b-7244d3f7f463 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:17.385918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-55a5f5a8-d144-4914-b93b-7244d3f7f463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:17.408801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:17.460878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 959/1929] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:17.482042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:17.484582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:17.555924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:17.571511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffed734-eac9-43b0-9a59-e9eb055e3b08 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:17.578118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:17.578764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:17.579092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:17.579396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:17.590517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:17.592016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:17.603568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 971/1930] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:17.885837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:17.886334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:17.886763 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:17.911937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:17.916038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:17.939221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3866c-247d-47c7-b043-fb58e7584f08 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:17.951344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:17.953501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:17.953995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:17.954607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] PATH_INFO: `/v3/projects/e6e75a99172649c6a87194f93c2f408c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:17.956051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e75a99172649c6a87194f93c2f408c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:17.957870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] MFA Rules not processed for user `9d67b1e32ef34deba8791de44bb35708`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:17.958369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:17.960540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:18.057884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e75a99172649c6a87194f93c2f408c', 'request_id': 'req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.066263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e75a99172649c6a87194f93c2f408c', 'request_id': 'req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.077652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e75a99172649c6a87194f93c2f408c', 'request_id': 'req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.087102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 960/1931] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:18.099561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:18.100838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] The token cache is being invalidate because project e6e75a99172649c6a87194f93c2f408c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:18.101400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:18.102048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.103706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.105794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 972/1932] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:17 2022] DELETE /identity/v3/projects/e6e75a99172649c6a87194f93c2f408c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:18.110799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:18.112308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:18.174421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c001a-e06c-409a-a627-8606f80113ea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:18.189335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cb4c0-4f6e-4d31-b7e1-a2eed54264e4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:18.194330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:18.194980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:18.195811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:18.196089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] PATH_INFO: `/v3/users/abf3cee12c8b48dc8597ad95fe512001` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:18.197277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf3cee12c8b48dc8597ad95fe512001)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:18.199231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:18.241196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:18.241987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:18.242351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:18.242688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:18.244891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:18.260573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf3cee12c8b48dc8597ad95fe512001'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.274411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf3cee12c8b48dc8597ad95fe512001'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.281382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:18.294236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:18.295189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf3cee12c8b48dc8597ad95fe512001'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.296174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf3cee12c8b48dc8597ad95fe512001'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.312481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf3cee12c8b48dc8597ad95fe512001'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.321393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 973/1933] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:18 2022] DELETE /identity/v3/users/abf3cee12c8b48dc8597ad95fe512001 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:18.350531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:18.352057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:18.366249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 961/1934] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:18.551856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:18.553370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:18.637564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d08fd-8a87-4ec4-a534-9ee37ce8d350 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:18.644937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:18.645997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:18.646470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:18.648986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] PATH_INFO: `/v3/users/eaf2af757aee4125a1536ed14b772b95` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:18.650443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf2af757aee4125a1536ed14b772b95)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:18.652934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:18.751317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:18.751601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf2af757aee4125a1536ed14b772b95'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.767424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf2af757aee4125a1536ed14b772b95'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.781367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e7819-6a2c-4e63-bc99-22c2d09ac447 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:18.791393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:18.792391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:18.792391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:18.792708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] PATH_INFO: `/v3/projects/764781db6e4e45d4b71bb00c131a9533` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:18.794056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] RBAC: Authorizing `identity:delete_project(project_id=764781db6e4e45d4b71bb00c131a9533)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:18.796185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:18.798165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf2af757aee4125a1536ed14b772b95'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.799577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf2af757aee4125a1536ed14b772b95'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.849530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e5e73a42-9132-4589-bb7e-7dc577b3e80e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf2af757aee4125a1536ed14b772b95'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.856704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 974/1935] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:18 2022] DELETE /identity/v3/users/eaf2af757aee4125a1536ed14b772b95 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:18.861199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:18.861486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:18.862135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:18.920706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:18.922258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:18.971316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764781db6e4e45d4b71bb00c131a9533', 'request_id': 'req-a07399e7-ab62-4e25-ad55-4edb368d203e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:18.996209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764781db6e4e45d4b71bb00c131a9533', 'request_id': 'req-a07399e7-ab62-4e25-ad55-4edb368d203e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.017292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764781db6e4e45d4b71bb00c131a9533', 'request_id': 'req-a07399e7-ab62-4e25-ad55-4edb368d203e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.027178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 975/1936] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:19.038013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.039969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.040322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] The token cache is being invalidate because project 764781db6e4e45d4b71bb00c131a9533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:19.040842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.041713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a07399e7-ab62-4e25-ad55-4edb368d203e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.044203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 962/1937] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:18 2022] DELETE /identity/v3/projects/764781db6e4e45d4b71bb00c131a9533 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:19.048742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557485d6-737e-4d82-b0d2-1de6dc182cb1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.049467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557485d6-737e-4d82-b0d2-1de6dc182cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.049974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557485d6-737e-4d82-b0d2-1de6dc182cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.052536 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:19.094641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-557485d6-737e-4d82-b0d2-1de6dc182cb1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:19.096797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-557485d6-737e-4d82-b0d2-1de6dc182cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.108102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 963/1938] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:19.116004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.116416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.117535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.149860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:19.150849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.163359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eb55d-8f2d-484d-843b-c9145aeef4e4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.169628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 964/1939] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:19.170430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.171237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.171575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.171941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.173677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.174039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.175092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.186938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedc90e-b0b5-4db6-be95-730afbdec619 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.187111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.188536 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.191561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.192214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.192491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.192762 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] PATH_INFO: `/v3/users/ec13ab2b6ecd483cac150babbfa6c6b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.193866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec13ab2b6ecd483cac150babbfa6c6b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.196224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.198137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 976/1940] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:19.198910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.208119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f05311-a8f1-4eab-8060-ca7c09d983d1 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.213641 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.214582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.214582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.214808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.215584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:19.215929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.217938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.234389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 977/1941] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:19.238778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.255116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd9b91-24db-4cba-a7d5-2a2391f85d9e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.257815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec13ab2b6ecd483cac150babbfa6c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.258601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec13ab2b6ecd483cac150babbfa6c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.262853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.263599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.263917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.264192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.264834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.266599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec13ab2b6ecd483cac150babbfa6c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.266878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.285017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec13ab2b6ecd483cac150babbfa6c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.286719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-09cba9b2-7428-4028-8b17-cf97d4f01b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec13ab2b6ecd483cac150babbfa6c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.301247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 978/1942] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:19 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:19.301935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 965/1943] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:19 2022] DELETE /identity/v3/users/ec13ab2b6ecd483cac150babbfa6c6b5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:19.305314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.306580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.306927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.307293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.307395 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.316528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:19.333114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:19.335427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.390270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfacb9-4d09-4f62-b390-18d5018fbabf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.409879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 966/1944] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:19.420414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.421872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.460028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.460707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.461571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.461865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.462502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.465425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.488784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13da8fb-c32f-41a9-ab24-5e38bd000aba None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.494330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 979/1945] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:19 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:19.495942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.496730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.497540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.497665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.497823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.498090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.509518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.511049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.511557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed454b72-7e11-41a1-be70-9eddd58e2e5d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.516592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.517232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.517515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.517792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] PATH_INFO: `/v3/projects/55d5ada1506c4ab4995a9377a6634cb1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.518919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d5ada1506c4ab4995a9377a6634cb1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.520879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.521777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 967/1946] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:19.522602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.528947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:19.534304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3173ca8-deee-48c3-8350-b660a77cff5f service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.540028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.540635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.540914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.541186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.541982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.544151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.556712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 968/1947] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:19.560787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.571789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c711f9-4531-48d6-b0a7-59cdc8776f1f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.598259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d5ada1506c4ab4995a9377a6634cb1', 'request_id': 'req-7787402b-530e-4712-8640-636aef5015cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.605276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d5ada1506c4ab4995a9377a6634cb1', 'request_id': 'req-7787402b-530e-4712-8640-636aef5015cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.613367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d5ada1506c4ab4995a9377a6634cb1', 'request_id': 'req-7787402b-530e-4712-8640-636aef5015cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.626269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] The token cache is being invalidate because project 55d5ada1506c4ab4995a9377a6634cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:19.626653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.627331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.628909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 980/1948] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:19 2022] DELETE /identity/v3/projects/55d5ada1506c4ab4995a9377a6634cb1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:19.633243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.634694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.653905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.654689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.655074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.655467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.656289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.658186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.669071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 969/1949] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:19.673681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.677164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.692163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09bc71-9df8-4930-9df4-3687c9f179ce admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.697667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a5371-b806-48ee-bed4-48e63307c474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.698350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5371-b806-48ee-bed4-48e63307c474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.698681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5371-b806-48ee-bed4-48e63307c474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.698972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5371-b806-48ee-bed4-48e63307c474 admin admin] PATH_INFO: `/v3/projects/0a7c966feb6e40d5bc6eb50c25d4ecde/users/e815949444dc4977b5a2e79fb6c7f0ba/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.708613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787402b-530e-4712-8640-636aef5015cc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:19.714180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:19.715439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.715846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.716216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] PATH_INFO: `/v3/users/436481482e104deaa6e25216f9414ee5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.717659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] RBAC: Authorizing `identity:delete_user(user_id=436481482e104deaa6e25216f9414ee5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.719711 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.719960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a5371-b806-48ee-bed4-48e63307c474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7c966feb6e40d5bc6eb50c25d4ecde, user_id=e815949444dc4977b5a2e79fb6c7f0ba, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:19.722617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a5371-b806-48ee-bed4-48e63307c474 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:19.743842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 970/1950] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:19 2022] PUT /identity/v3/projects/0a7c966feb6e40d5bc6eb50c25d4ecde/users/e815949444dc4977b5a2e79fb6c7f0ba/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:19.751840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:19.752453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:19.752793 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:19.774593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:19.776105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436481482e104deaa6e25216f9414ee5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.785710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436481482e104deaa6e25216f9414ee5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.788512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] MFA Rules not processed for user `e815949444dc4977b5a2e79fb6c7f0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:19.794149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.803695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436481482e104deaa6e25216f9414ee5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.805821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436481482e104deaa6e25216f9414ee5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.817605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436481482e104deaa6e25216f9414ee5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:19.828254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 981/1951] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:19 2022] DELETE /identity/v3/users/436481482e104deaa6e25216f9414ee5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:19.869470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.871060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:19.910670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 971/1952] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:19.914203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:19.915621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:19.925400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.045552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1873b-ae14-4424-abb9-07ba2f72632b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.059961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.060942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.061493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.062230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] PATH_INFO: `/v3/projects/d742327ee5064515ac02daa3671cf777` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.063620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d742327ee5064515ac02daa3671cf777)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.065892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.083178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c54a15-1004-472a-9de8-be20a0119f5f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.095705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.098562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.100298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.100753 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.102794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.107593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.157609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 972/1953] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:19 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:20.158589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.161525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.186425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd742327ee5064515ac02daa3671cf777', 'request_id': 'req-5ff7e7f3-2415-474e-98fc-2815614bd06c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:20.201846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd742327ee5064515ac02daa3671cf777', 'request_id': 'req-5ff7e7f3-2415-474e-98fc-2815614bd06c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:20.213045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd742327ee5064515ac02daa3671cf777', 'request_id': 'req-5ff7e7f3-2415-474e-98fc-2815614bd06c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:20.229398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] The token cache is being invalidate because project d742327ee5064515ac02daa3671cf777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:20.229739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:20.231146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:20.233186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 982/1954] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:19 2022] DELETE /identity/v3/projects/d742327ee5064515ac02daa3671cf777 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:20.234004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.234500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733eaa9-7e2c-45c6-a376-9414839e695a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.240207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.293808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:20.310557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.311107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.311386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.311712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.313949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.360044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7e7f3-2415-474e-98fc-2815614bd06c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.368473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf56aa-57e3-433f-8c8c-6cfcf07e5504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.369515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf56aa-57e3-433f-8c8c-6cfcf07e5504 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.370108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf56aa-57e3-433f-8c8c-6cfcf07e5504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.370532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf56aa-57e3-433f-8c8c-6cfcf07e5504 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.371425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf56aa-57e3-433f-8c8c-6cfcf07e5504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.375155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf56aa-57e3-433f-8c8c-6cfcf07e5504 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.413605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 983/1955] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:20 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:20.416850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaae18-6ff3-40a7-8608-0508b48a5df8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.418767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaae18-6ff3-40a7-8608-0508b48a5df8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.419059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaae18-6ff3-40a7-8608-0508b48a5df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.426847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.431843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.463052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 973/1956] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:20.463837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7cdaae18-6ff3-40a7-8608-0508b48a5df8 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:20.464357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.466758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7cdaae18-6ff3-40a7-8608-0508b48a5df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.477902 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e87e5-7b10-41cc-ab93-718efd28521a service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.483744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.484861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.485276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.485681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.486418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.488265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.491300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 984/1957] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:20.500528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95d40-d689-4dfe-bf80-cdcc0b36079d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.501464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95d40-d689-4dfe-bf80-cdcc0b36079d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.501761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95d40-d689-4dfe-bf80-cdcc0b36079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.506624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 974/1958] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:20.508090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.521361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc097ca7-4d3d-48d6-ab87-5085e34663fe admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.527918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.529069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.529499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.529934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] PATH_INFO: `/v3/projects/e3d299627f5246c3b85e261b061aa4b8/users/2aa1695de9b3447eae007a0123508c70/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.550130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d299627f5246c3b85e261b061aa4b8, user_id=2aa1695de9b3447eae007a0123508c70, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.553758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9fc95d40-d689-4dfe-bf80-cdcc0b36079d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:20.555154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.556765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9fc95d40-d689-4dfe-bf80-cdcc0b36079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.589545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 975/1959] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:20 2022] PUT /identity/v3/projects/e3d299627f5246c3b85e261b061aa4b8/users/2aa1695de9b3447eae007a0123508c70/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:20.593183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 985/1960] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:20.595891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.601299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95d40-d689-4dfe-bf80-cdcc0b36079d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.642965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95d40-d689-4dfe-bf80-cdcc0b36079d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.679883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166a1a-3d18-48d9-8710-d1a54b8e1eca admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.683378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:20.685980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.687179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.687565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.687937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.688778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.690791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.702875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 976/1961] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:20.718467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.755603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766daf8-7560-4a27-b0a2-5e4b429d49fc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.760022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b3e4e-5873-4291-93f3-dbaf5673b28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.760665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b3e4e-5873-4291-93f3-dbaf5673b28c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.760940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b3e4e-5873-4291-93f3-dbaf5673b28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.761205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b3e4e-5873-4291-93f3-dbaf5673b28c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.761927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b3e4e-5873-4291-93f3-dbaf5673b28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:20.762142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b3e4e-5873-4291-93f3-dbaf5673b28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.763789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b3e4e-5873-4291-93f3-dbaf5673b28c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.770027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.770576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.770851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.771123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] PATH_INFO: `/v3/projects/e3d299627f5246c3b85e261b061aa4b8/users/2aa1695de9b3447eae007a0123508c70/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.773242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d299627f5246c3b85e261b061aa4b8, user_id=2aa1695de9b3447eae007a0123508c70, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.775918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.787890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 977/1962] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:20 2022] PUT /identity/v3/projects/e3d299627f5246c3b85e261b061aa4b8/users/2aa1695de9b3447eae007a0123508c70/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:20.797077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 986/1963] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:20.799715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:20.801667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.805425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49914385-a2c8-413d-a1fb-0aa74d67da94 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.805632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49914385-a2c8-413d-a1fb-0aa74d67da94 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.805914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49914385-a2c8-413d-a1fb-0aa74d67da94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.848361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-49914385-a2c8-413d-a1fb-0aa74d67da94 None None] MFA Rules not processed for user `2aa1695de9b3447eae007a0123508c70`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:20.851070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-49914385-a2c8-413d-a1fb-0aa74d67da94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:20.872788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:20.926212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888448c4-e745-4be9-b217-bb3a008c97d8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:20.933583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:20.936852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:20.937141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:20.937415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] PATH_INFO: `/v3/projects/3097fa00940043ddbea0805963ce9328` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:20.939994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3097fa00940043ddbea0805963ce9328)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:20.942157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:20.970120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 987/1964] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:20.970120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49914385-a2c8-413d-a1fb-0aa74d67da94 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.000680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49914385-a2c8-413d-a1fb-0aa74d67da94 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.013394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3097fa00940043ddbea0805963ce9328', 'request_id': 'req-3e3f0821-1a6f-4559-877f-46b4235cf844'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:21.024007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3097fa00940043ddbea0805963ce9328', 'request_id': 'req-3e3f0821-1a6f-4559-877f-46b4235cf844'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:21.037237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3097fa00940043ddbea0805963ce9328', 'request_id': 'req-3e3f0821-1a6f-4559-877f-46b4235cf844'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:21.063715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] The token cache is being invalidate because project 3097fa00940043ddbea0805963ce9328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:21.064278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:21.065175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:21.067337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 978/1965] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:20 2022] DELETE /identity/v3/projects/3097fa00940043ddbea0805963ce9328 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:21.068355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.072288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:21.095925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4ae22-4031-4c36-b42b-a97e69646c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.097739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4ae22-4031-4c36-b42b-a97e69646c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.097739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4ae22-4031-4c36-b42b-a97e69646c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.097739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4ae22-4031-4c36-b42b-a97e69646c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.098650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4ae22-4031-4c36-b42b-a97e69646c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.100856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4ae22-4031-4c36-b42b-a97e69646c56 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.172445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 988/1966] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:20 2022] POST /identity/v3/projects => generated 369 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:21.174145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.174513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.174874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.199529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0821-1a6f-4559-877f-46b4235cf844 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.208708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:21.210089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:21.236185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:21.325280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.325656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 989/1967] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:21.326650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.327045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.327426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.330323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:21.333352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.334934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:21.348146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8d5d8-3fde-479e-9dd3-0631d2b406f4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.353606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676db0e-af40-4744-93ad-bc871cf7350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.354237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676db0e-af40-4744-93ad-bc871cf7350a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.354588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676db0e-af40-4744-93ad-bc871cf7350a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.354859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676db0e-af40-4744-93ad-bc871cf7350a admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.355499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676db0e-af40-4744-93ad-bc871cf7350a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.357628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676db0e-af40-4744-93ad-bc871cf7350a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.379845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:21.390138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 990/1968] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:21 2022] POST /identity/v3/users => generated 285 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:21.390138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afde88f-b4f0-46cb-af9c-ffec6acb62e0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.390138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afde88f-b4f0-46cb-af9c-ffec6acb62e0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.390138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afde88f-b4f0-46cb-af9c-ffec6acb62e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.409597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.411051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.414798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1afde88f-b4f0-46cb-af9c-ffec6acb62e0 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:21.416401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1afde88f-b4f0-46cb-af9c-ffec6acb62e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:21.422553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 979/1969] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:21.423239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.431013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 991/1970] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:21.432902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afde88f-b4f0-46cb-af9c-ffec6acb62e0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.436934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9247e-462f-451a-a4df-4aba232463c7 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.442811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.443652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.444097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.444208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afde88f-b4f0-46cb-af9c-ffec6acb62e0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.444601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.445460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.447225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.449710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.450730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.451031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.451286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.452123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:21.452440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.454867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.459836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 980/1971] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:21.463401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.473397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 992/1972] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:21.474290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.483574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f42e6-4df9-4fd4-b48f-bcf32d54f642 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.484951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15048709-1452-4584-b0b2-ed23381e7cdc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.489765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.489947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.490400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.490586 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.490672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.490873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.490983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.491158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.491617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.491946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.493642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.494048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.509124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 981/1973] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:21.514866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.527545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c468728-f62f-4d18-8c94-7770b04bb3aa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.534653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 993/1974] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:21 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:21.535889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08ac83-ec0c-4b31-aeea-bd01f3646132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.536704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08ac83-ec0c-4b31-aeea-bd01f3646132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.537063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08ac83-ec0c-4b31-aeea-bd01f3646132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.538193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08ac83-ec0c-4b31-aeea-bd01f3646132 admin admin] PATH_INFO: `/v3/projects/e6712d46aeb748ce94acb8b8d0d6c927/users/cbb8df8fb5d447f3aa62939d6958c069/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.538318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.557781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08ac83-ec0c-4b31-aeea-bd01f3646132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6712d46aeb748ce94acb8b8d0d6c927, user_id=cbb8df8fb5d447f3aa62939d6958c069, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.560657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08ac83-ec0c-4b31-aeea-bd01f3646132 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.582200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 982/1975] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:21 2022] PUT /identity/v3/projects/e6712d46aeb748ce94acb8b8d0d6c927/users/cbb8df8fb5d447f3aa62939d6958c069/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:21.593164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.593553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.593927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.631603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] MFA Rules not processed for user `cbb8df8fb5d447f3aa62939d6958c069`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:21.632527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:21.683816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d9c70-d2c7-43cd-8bf2-ef2a9f0231fb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.712106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 983/1976] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:21.723545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.725299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:21.776557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.777283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.778319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.778672 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.779393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.781566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.809066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 994/1977] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:21 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:21.813163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.818805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:21.820520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5373a53-5f19-43e5-ac53-7dde312d06d8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.825453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.826009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.826314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.826589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.831542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a250c-682b-495b-8f2a-f42e092c1bd9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.838632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.840402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.853146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 984/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:21.853951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.854590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.854873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.855151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.855832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.857440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.883897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 995/1979] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:21.888315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.904821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4420844-71f5-438d-935b-a2853efedbfe service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.920398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdd897-1f65-41ee-b5ba-fb9b90ace947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.921049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd897-1f65-41ee-b5ba-fb9b90ace947 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.921349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd897-1f65-41ee-b5ba-fb9b90ace947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.921604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd897-1f65-41ee-b5ba-fb9b90ace947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:21.922310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdd897-1f65-41ee-b5ba-fb9b90ace947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:21.924112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdd897-1f65-41ee-b5ba-fb9b90ace947 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:21.941089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 985/1980] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:21.941709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:21.958407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1adac9-7731-4095-be79-47a660db32f3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:21.970183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:21.971261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:21.971553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:21.971858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] PATH_INFO: `/v3/projects/da045ce12c93458691453eb61b5fee28/users/355fa8918bd84699838eb10756419f75/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.006878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da045ce12c93458691453eb61b5fee28, user_id=355fa8918bd84699838eb10756419f75, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.011406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.044259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 996/1981] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:21 2022] PUT /identity/v3/projects/da045ce12c93458691453eb61b5fee28/users/355fa8918bd84699838eb10756419f75/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:22.051843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11334b5a-7734-441e-8f9e-2f73ed5c5009 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.052660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11334b5a-7734-441e-8f9e-2f73ed5c5009 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.053210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11334b5a-7734-441e-8f9e-2f73ed5c5009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.080641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:22.093278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-11334b5a-7734-441e-8f9e-2f73ed5c5009 None None] MFA Rules not processed for user `355fa8918bd84699838eb10756419f75`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:22.096679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-11334b5a-7734-441e-8f9e-2f73ed5c5009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:22.205788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 986/1982] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:22.210411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.339724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61d85-7ae5-460e-86ab-75fe1b97fa0c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.345787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:22.346530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.346877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.347124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.347888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.357190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.409553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 997/1983] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:22 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:22.410816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11334b5a-7734-441e-8f9e-2f73ed5c5009 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.428008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11334b5a-7734-441e-8f9e-2f73ed5c5009 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.575922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:22.575922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.575922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.575922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.576689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.578017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.608809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.610562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:22.614275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 987/1984] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:22 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:22.622700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.634634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b109613-5da0-4a98-a984-1f3487f81c09 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.637108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac00e-d5c1-42ac-a266-33d052996485 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.640025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:22.640846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.641281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.641719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] PATH_INFO: `/v3/users/e815949444dc4977b5a2e79fb6c7f0ba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.641989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:22.642593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.642869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.643124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e815949444dc4977b5a2e79fb6c7f0ba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.643236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.643771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.645363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.645732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.664175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 988/1985] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:22.667448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.678607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685138f-7418-4dc1-8140-647bdcdeb280 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.685654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:22.686473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.686930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.687200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] PATH_INFO: `/v3/projects/011583237877414390830d9a25994ffe/users/7b5e2c1d4e744c01870faa11f34d43b4/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.707172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011583237877414390830d9a25994ffe, user_id=7b5e2c1d4e744c01870faa11f34d43b4, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.711449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.725516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e815949444dc4977b5a2e79fb6c7f0ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:22.731882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 989/1986] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:22 2022] PUT /identity/v3/projects/011583237877414390830d9a25994ffe/users/7b5e2c1d4e744c01870faa11f34d43b4/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:22.737561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.738142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e815949444dc4977b5a2e79fb6c7f0ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:22.755409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e815949444dc4977b5a2e79fb6c7f0ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:22.757557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e815949444dc4977b5a2e79fb6c7f0ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:22.765154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-923a5114-5698-4361-a7c9-e0235a00c736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e815949444dc4977b5a2e79fb6c7f0ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:22.774073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 998/1987] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:22 2022] DELETE /identity/v3/users/e815949444dc4977b5a2e79fb6c7f0ba => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:22.778308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.778596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.778877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.797311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:22.799725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:22.801003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:22.811189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbf0c9-6388-4b7e-ad0f-da4c3d17b317 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.823009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:22.875986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 999/1988] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:22.881236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:22.881845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.882119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.882597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.883114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.883562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.886125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.887813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:22.898989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 990/1989] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:22.900198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:22.901954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:22.916187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd92876-8fed-44ba-86fd-eeb6e5be6802 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.921362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab657bc-1014-412b-b8bc-d2f306c6c7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:22.922029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab657bc-1014-412b-b8bc-d2f306c6c7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.922311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab657bc-1014-412b-b8bc-d2f306c6c7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.922610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab657bc-1014-412b-b8bc-d2f306c6c7c7 admin admin] PATH_INFO: `/v3/projects/011583237877414390830d9a25994ffe/users/7b5e2c1d4e744c01870faa11f34d43b4/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.924583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab657bc-1014-412b-b8bc-d2f306c6c7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011583237877414390830d9a25994ffe, user_id=7b5e2c1d4e744c01870faa11f34d43b4, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:22.927839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab657bc-1014-412b-b8bc-d2f306c6c7c7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:22.939116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 991/1990] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:22 2022] PUT /identity/v3/projects/011583237877414390830d9a25994ffe/users/7b5e2c1d4e744c01870faa11f34d43b4/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:22.951404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:22.951985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:22.952268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:22.966166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ab16f-e648-4c09-b4f4-623d4cf6b0dc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:22.983250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] MFA Rules not processed for user `7b5e2c1d4e744c01870faa11f34d43b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:22.985356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:23.024826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:23.025445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:23.025727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:23.026007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:23.103213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 992/1991] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:23.104237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:23.105905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:23.123600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:23.126863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:23.153545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1000/1992] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:23.176170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac8790-cbc9-4187-9ef7-f3446d0c786c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:23.182442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:23.183075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:23.183641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:23.183953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:23.196410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:23.198022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:23.211939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 993/1993] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:23.977482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:23.978892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:23.992944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7215c1-bb36-42bb-a9b9-05ac071b6268 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:23.998082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:23.998719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:23.999041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:23.999331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] PATH_INFO: `/v3/projects/0a7c966feb6e40d5bc6eb50c25d4ecde` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:24.000959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7c966feb6e40d5bc6eb50c25d4ecde)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:24.002900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:24.060097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:24.078917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7c966feb6e40d5bc6eb50c25d4ecde', 'request_id': 'req-d9719083-c40d-4275-b4cb-4e7f5646a341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.086274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7c966feb6e40d5bc6eb50c25d4ecde', 'request_id': 'req-d9719083-c40d-4275-b4cb-4e7f5646a341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.094701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7c966feb6e40d5bc6eb50c25d4ecde', 'request_id': 'req-d9719083-c40d-4275-b4cb-4e7f5646a341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.108507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] The token cache is being invalidate because project 0a7c966feb6e40d5bc6eb50c25d4ecde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:24.108796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.109588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.111861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1001/1994] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:23 2022] DELETE /identity/v3/projects/0a7c966feb6e40d5bc6eb50c25d4ecde => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:24.115993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:24.128547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc4162-8e3b-433c-95aa-7964f070d175 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:24.138082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:24.169403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:24.170816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:24.237075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:24.237896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:24.238185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:24.238460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] PATH_INFO: `/v3/users/2aa1695de9b3447eae007a0123508c70` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:24.240266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa1695de9b3447eae007a0123508c70)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:24.246170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:24.269270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719083-c40d-4275-b4cb-4e7f5646a341 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:24.276169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:24.276958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:24.277340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:24.278806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] PATH_INFO: `/v3/users/355fa8918bd84699838eb10756419f75` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:24.279382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=355fa8918bd84699838eb10756419f75)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:24.284121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:24.361279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa1695de9b3447eae007a0123508c70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.362835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa1695de9b3447eae007a0123508c70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.380038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa1695de9b3447eae007a0123508c70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.382582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '355fa8918bd84699838eb10756419f75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.398955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '355fa8918bd84699838eb10756419f75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.418870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa1695de9b3447eae007a0123508c70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.421581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a5d98225-cb1e-4597-930d-fba40ff0ca80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa1695de9b3447eae007a0123508c70'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.426225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '355fa8918bd84699838eb10756419f75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.427775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '355fa8918bd84699838eb10756419f75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.445390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '355fa8918bd84699838eb10756419f75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:24.451442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 994/1995] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:24 2022] DELETE /identity/v3/users/2aa1695de9b3447eae007a0123508c70 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:24.460204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1002/1996] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:24 2022] DELETE /identity/v3/users/355fa8918bd84699838eb10756419f75 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:24.465623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:24.466035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:24.466382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:24.510452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:24.511705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:24.678266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 995/1997] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:24.684887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:24.686350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:24.741412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:24.791066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75197cac-d9c1-4645-ad52-e0995dfc9ff6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:24.797179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6316a-9aa5-4e4b-ad9f-8aebe78fbf18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:24.797841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6316a-9aa5-4e4b-ad9f-8aebe78fbf18 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:24.798171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6316a-9aa5-4e4b-ad9f-8aebe78fbf18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:24.799573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6316a-9aa5-4e4b-ad9f-8aebe78fbf18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:24.811886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6316a-9aa5-4e4b-ad9f-8aebe78fbf18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:24.813864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6316a-9aa5-4e4b-ad9f-8aebe78fbf18 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:24.829016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1003/1998] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:25.075416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:25.077596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:25.115845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8a29c-e1d4-4e35-ac47-6fd2f1e54468 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:25.143929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:25.145490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:25.145949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:25.146892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] PATH_INFO: `/v3/projects/e3d299627f5246c3b85e261b061aa4b8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:25.149778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d299627f5246c3b85e261b061aa4b8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:25.153003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:25.229578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:25.260144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d299627f5246c3b85e261b061aa4b8', 'request_id': 'req-527e5182-c011-497d-89d0-06f2601e4f76'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.277151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d299627f5246c3b85e261b061aa4b8', 'request_id': 'req-527e5182-c011-497d-89d0-06f2601e4f76'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.288717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d299627f5246c3b85e261b061aa4b8', 'request_id': 'req-527e5182-c011-497d-89d0-06f2601e4f76'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.316446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] The token cache is being invalidate because project e3d299627f5246c3b85e261b061aa4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:25.316731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.318145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-527e5182-c011-497d-89d0-06f2601e4f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.322520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 996/1999] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:25 2022] DELETE /identity/v3/projects/e3d299627f5246c3b85e261b061aa4b8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:25.327495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:25.328679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:25.328985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:25.352001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:25.354036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:25.462029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597510d-cf88-4810-8387-84a1bf28ba23 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:25.462386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597510d-cf88-4810-8387-84a1bf28ba23 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:25.462733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597510d-cf88-4810-8387-84a1bf28ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:25.467339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1004/2000] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:25.498932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7597510d-cf88-4810-8387-84a1bf28ba23 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:25.503263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7597510d-cf88-4810-8387-84a1bf28ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:25.516244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:25.518025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:25.524108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 997/2001] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:25.527332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597510d-cf88-4810-8387-84a1bf28ba23 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:25.546243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597510d-cf88-4810-8387-84a1bf28ba23 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:25.567673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349a192-3648-47ed-9155-f66a0de05a38 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:25.597492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:25.598208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:25.598449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:25.599293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:25.599998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:25.600114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:25.600662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:25.601500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] PATH_INFO: `/v3/projects/da045ce12c93458691453eb61b5fee28` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:25.601610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:25.601851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:25.603981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da045ce12c93458691453eb61b5fee28)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:25.607342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:25.648248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:25.648780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 998/2002] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:25.648780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:25.664884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb56056-2c3a-487b-aa59-de5961c97290 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:25.680121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:25.682181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:25.682181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:25.682181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:25.682770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:25.684269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:25.780752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 999/2003] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:25 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:25.784805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:25.843738 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da045ce12c93458691453eb61b5fee28', 'request_id': 'req-0166e384-6812-4cab-ada1-becdfd685be3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.850749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da045ce12c93458691453eb61b5fee28', 'request_id': 'req-0166e384-6812-4cab-ada1-becdfd685be3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.865042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da045ce12c93458691453eb61b5fee28', 'request_id': 'req-0166e384-6812-4cab-ada1-becdfd685be3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.889246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] The token cache is being invalidate because project da045ce12c93458691453eb61b5fee28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:25.889547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.890219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:25.892117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1005/2004] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:25 2022] DELETE /identity/v3/projects/da045ce12c93458691453eb61b5fee28 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:25.896017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:25.899864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:25.931081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf638406-e3c3-46f7-b4d9-4d453b47de97 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.027864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e384-6812-4cab-ada1-becdfd685be3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.046262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.046262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.046262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.046262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] PATH_INFO: `/v3/users/7b5e2c1d4e744c01870faa11f34d43b4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.046727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.046846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5e2c1d4e744c01870faa11f34d43b4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.046938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.046938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.046938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.046938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.047237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:26.047325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:26.048413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:26.088855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1000/2005] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:25 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:26.091347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:26.098651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:26.117174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172c1b4-186c-4a27-bbb3-616571d2ed27 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.124374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.125153 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.125570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.125918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] PATH_INFO: `/v3/users/cbb8df8fb5d447f3aa62939d6958c069` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.127325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb8df8fb5d447f3aa62939d6958c069)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.129434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:26.133740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:26.150059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5e2c1d4e744c01870faa11f34d43b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.160343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5e2c1d4e744c01870faa11f34d43b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.176235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5e2c1d4e744c01870faa11f34d43b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.177253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5e2c1d4e744c01870faa11f34d43b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.186089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5e2c1d4e744c01870faa11f34d43b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.194348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1006/2006] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:25 2022] DELETE /identity/v3/users/7b5e2c1d4e744c01870faa11f34d43b4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:26.196017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:26.197992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:26.223096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb8df8fb5d447f3aa62939d6958c069'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.224335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb8df8fb5d447f3aa62939d6958c069'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.239800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb8df8fb5d447f3aa62939d6958c069'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.263272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb8df8fb5d447f3aa62939d6958c069'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.265067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6914779f-0c64-4a45-9b6e-d008a4f88046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb8df8fb5d447f3aa62939d6958c069'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:26.287091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9603fe-93b0-4611-93db-97ea7fd26604 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.293916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.294212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1001/2007] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:26 2022] DELETE /identity/v3/users/cbb8df8fb5d447f3aa62939d6958c069 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:26.295425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.295882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.297283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.297963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.298954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.299348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.299660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.300388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:26.320731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1007/2008] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:26.328020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:26.336358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:26.348012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:26.351327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:26.370899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0645b7e-f8e5-4fde-bda6-cf1369f4fcaf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.490282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1002/2009] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:26.510628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:26.512379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:26.514566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.515889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.516187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.516623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.517294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.518943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:26.547212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1008/2010] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:26.549594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:26.560997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab0bf-5f16-4c3b-b090-6c80c4e5840e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.568442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bd872-ec2c-4221-bd49-61199462e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.569185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bd872-ec2c-4221-bd49-61199462e93a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.570340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bd872-ec2c-4221-bd49-61199462e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.570340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bd872-ec2c-4221-bd49-61199462e93a admin admin] PATH_INFO: `/v3/projects/6a927d4407b14bfa91f2279f18276e54/users/fc7d3e371f4f4a6182aab23747ad6a89/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.585701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:26.589122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bd872-ec2c-4221-bd49-61199462e93a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a927d4407b14bfa91f2279f18276e54, user_id=fc7d3e371f4f4a6182aab23747ad6a89, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.593514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bd872-ec2c-4221-bd49-61199462e93a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:26.611659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc379fc5-2cec-4ad0-80c6-eb6fddc15ecc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.633135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1009/2011] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:26 2022] PUT /identity/v3/projects/6a927d4407b14bfa91f2279f18276e54/users/fc7d3e371f4f4a6182aab23747ad6a89/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:26.637504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.637790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.638094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.697290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:26.701498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] MFA Rules not processed for user `fc7d3e371f4f4a6182aab23747ad6a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:26.702976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:26.725549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.726615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.727090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.727559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.824115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1010/2012] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:26.831176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:26.835063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:26.953116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.954989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:26.966339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1003/2013] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:26.976960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:26.979049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad4c9c-6d33-4cb3-9905-4659c02bc3e8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:26.988485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:26.989208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:26.989604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:26.989879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] PATH_INFO: `/v3/projects/011583237877414390830d9a25994ffe` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:26.992342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011583237877414390830d9a25994ffe)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:26.996891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.004779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eda39-a421-4271-a7e6-2dee9bb96684 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.018202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.018832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.019117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.019392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.020077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.022660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.079918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011583237877414390830d9a25994ffe', 'request_id': 'req-86f26124-bc2f-43ec-a993-a8524f2040e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.082958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1004/2014] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:26 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:27.084025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.089963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011583237877414390830d9a25994ffe', 'request_id': 'req-86f26124-bc2f-43ec-a993-a8524f2040e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.103507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011583237877414390830d9a25994ffe', 'request_id': 'req-86f26124-bc2f-43ec-a993-a8524f2040e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.125799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] The token cache is being invalidate because project 011583237877414390830d9a25994ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:27.126040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.126763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-86f26124-bc2f-43ec-a993-a8524f2040e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.128566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1011/2015] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:26 2022] DELETE /identity/v3/projects/011583237877414390830d9a25994ffe => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:27.134474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c86e3b-4532-4537-94a6-dfedc852d298 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.134841 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c86e3b-4532-4537-94a6-dfedc852d298 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.135314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c86e3b-4532-4537-94a6-dfedc852d298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.161877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:27.180453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-09c86e3b-4532-4537-94a6-dfedc852d298 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:27.182797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-09c86e3b-4532-4537-94a6-dfedc852d298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:27.202533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1012/2016] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:27.206675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698dbe5-ec40-4eb7-b501-dccda4ce3622 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.211662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7a1f-b087-4c71-8df9-6bc80b359d41 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.212079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7a1f-b087-4c71-8df9-6bc80b359d41 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.212445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7a1f-b087-4c71-8df9-6bc80b359d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.221725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.222353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.222630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.222898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.223792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.225522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.263349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-000b7a1f-b087-4c71-8df9-6bc80b359d41 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:27.266383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-000b7a1f-b087-4c71-8df9-6bc80b359d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:27.271792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1005/2017] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:27.275869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.277795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:27.291904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1013/2018] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:27.293189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b7a1f-b087-4c71-8df9-6bc80b359d41 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.297010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2a115-f358-45be-bf79-af1413a2d71e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.307261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.307935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.308357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.308525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b7a1f-b087-4c71-8df9-6bc80b359d41 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.308651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.309295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.311403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.316067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.317024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.317441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.317863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.318965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:27.319555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.321575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.329050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1006/2019] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:27.332978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.341828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1014/2020] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:27.346264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.349467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f1b24-4615-4bd9-ae65-63c6de545b1e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.358044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.358764 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.359095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a1ff-0789-4af5-85ca-634056a1f69a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.359233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.359503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.360469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.362267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.365202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.365981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.366380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.367475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.368446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.370902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.393539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1007/2021] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:27.396901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.421445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9f753-eb80-452e-b521-b999a14f5730 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.428913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf9fc-a7e5-491c-a569-9c7696c617bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.430173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf9fc-a7e5-491c-a569-9c7696c617bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.430676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf9fc-a7e5-491c-a569-9c7696c617bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.431157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf9fc-a7e5-491c-a569-9c7696c617bb admin admin] PATH_INFO: `/v3/projects/73802156e51542eeb539235cee007650/users/6d187d7a78ef4c73b6df2b1b50ea6b2b/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.433301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1015/2022] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:27.438956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.450974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebf9fc-a7e5-491c-a569-9c7696c617bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=73802156e51542eeb539235cee007650, user_id=6d187d7a78ef4c73b6df2b1b50ea6b2b, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.454532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebf9fc-a7e5-491c-a569-9c7696c617bb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.485901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1008/2023] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:27 2022] PUT /identity/v3/projects/73802156e51542eeb539235cee007650/users/6d187d7a78ef4c73b6df2b1b50ea6b2b/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:27.500733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e58c-94d1-49a1-bd79-51dfab06fa76 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.501998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e58c-94d1-49a1-bd79-51dfab06fa76 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.502883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e58c-94d1-49a1-bd79-51dfab06fa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.551090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46982960-eb73-4d44-96a9-db9576c61960 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.566886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:27.589839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-67f4e58c-94d1-49a1-bd79-51dfab06fa76 None None] MFA Rules not processed for user `6d187d7a78ef4c73b6df2b1b50ea6b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:27.591231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-67f4e58c-94d1-49a1-bd79-51dfab06fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:27.623344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.624157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.624548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.625203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.625732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.635898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.717383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1016/2024] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:27.720320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.722125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:27.737406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0722c-a216-40e4-a43d-2732096c5a9e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.750093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.750927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.751132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.751410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] PATH_INFO: `/v3/projects/e6712d46aeb748ce94acb8b8d0d6c927` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.762598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6712d46aeb748ce94acb8b8d0d6c927)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.764891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.783675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1009/2025] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:27.784561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e58c-94d1-49a1-bd79-51dfab06fa76 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.809271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e58c-94d1-49a1-bd79-51dfab06fa76 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.820642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.821027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.821308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.821894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.822231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.824236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.834827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1010/2026] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:27.836217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.846920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee0623-ff2f-436e-b6e0-02631588c6c5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:27.848310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6712d46aeb748ce94acb8b8d0d6c927', 'request_id': 'req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.849933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:27.858236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6712d46aeb748ce94acb8b8d0d6c927', 'request_id': 'req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.870179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6712d46aeb748ce94acb8b8d0d6c927', 'request_id': 'req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.889256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] The token cache is being invalidate because project e6712d46aeb748ce94acb8b8d0d6c927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:27.889461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.890461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:27.893948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1017/2027] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:27 2022] DELETE /identity/v3/projects/e6712d46aeb748ce94acb8b8d0d6c927 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:27.894647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:27.896245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:27.914011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21537da5-bd83-4d6d-8b6f-2f366d74fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:27.914633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537da5-bd83-4d6d-8b6f-2f366d74fd36 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.914909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537da5-bd83-4d6d-8b6f-2f366d74fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.915179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537da5-bd83-4d6d-8b6f-2f366d74fd36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:27.915969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21537da5-bd83-4d6d-8b6f-2f366d74fd36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:27.918892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21537da5-bd83-4d6d-8b6f-2f366d74fd36 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:27.955109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1011/2028] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:27.955109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:27.955109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:27.955109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.009332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:28.010487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:28.029038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c1ffd-be3d-45c2-bfa9-9e7e42888690 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.178331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1012/2029] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:28.179898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.190455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:28.198925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda19791-2ea8-4458-bac3-a13f323990fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.199480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda19791-2ea8-4458-bac3-a13f323990fb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.199808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda19791-2ea8-4458-bac3-a13f323990fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.200081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda19791-2ea8-4458-bac3-a13f323990fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.200766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda19791-2ea8-4458-bac3-a13f323990fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.205316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda19791-2ea8-4458-bac3-a13f323990fb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.219745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b83ac-2efd-4d18-bda5-19a7326fa180 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.226529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1018/2030] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:28.227875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cdf71-0517-47c1-8ad2-b480ae5dbd70 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.228293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cdf71-0517-47c1-8ad2-b480ae5dbd70 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.228567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cdf71-0517-47c1-8ad2-b480ae5dbd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.234631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.235625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.235939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.236210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.236849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.240233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.268263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:28.272134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bb8cdf71-0517-47c1-8ad2-b480ae5dbd70 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:28.273508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb8cdf71-0517-47c1-8ad2-b480ae5dbd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:28.287563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1013/2031] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:28 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:28.288964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.301339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1019/2032] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:28.303856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cdf71-0517-47c1-8ad2-b480ae5dbd70 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.312450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de3b37-8f28-4e3f-a3e1-1dbc6e123fb0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.323042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cdf71-0517-47c1-8ad2-b480ae5dbd70 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.326975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.327995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.328628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.329187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] PATH_INFO: `/v3/projects/fd0ac8be80ab485a8724cc74b3d50f8f/users/8e212eb6f9c24bc1af28795e36494e90/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.344937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.346196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.347052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.347432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.348421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.353469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.373629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0ac8be80ab485a8724cc74b3d50f8f, user_id=8e212eb6f9c24bc1af28795e36494e90, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.394640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.409277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1020/2033] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:28.410314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.422480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6eed6-84f1-44ee-b44c-a41c890823b0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.427698 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078efb3-44c1-49d0-9ff3-a8965e70aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.428346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078efb3-44c1-49d0-9ff3-a8965e70aec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.428643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078efb3-44c1-49d0-9ff3-a8965e70aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.429714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078efb3-44c1-49d0-9ff3-a8965e70aec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.430669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078efb3-44c1-49d0-9ff3-a8965e70aec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:28.431003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078efb3-44c1-49d0-9ff3-a8965e70aec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.432731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078efb3-44c1-49d0-9ff3-a8965e70aec8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.450764 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1014/2034] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:28 2022] PUT /identity/v3/projects/fd0ac8be80ab485a8724cc74b3d50f8f/users/8e212eb6f9c24bc1af28795e36494e90/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:28.451686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.478300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1021/2035] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:28.478300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55c7c9-2c64-4df3-907f-b62f485dda54 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.478300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55c7c9-2c64-4df3-907f-b62f485dda54 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.478300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55c7c9-2c64-4df3-907f-b62f485dda54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.533982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab9aa1-6b45-4837-a7c2-fa65d88b789d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.535095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d55c7c9-2c64-4df3-907f-b62f485dda54 None None] MFA Rules not processed for user `8e212eb6f9c24bc1af28795e36494e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:28.536800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d55c7c9-2c64-4df3-907f-b62f485dda54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:28.547494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.548223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.548510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.548790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] PATH_INFO: `/v3/projects/898f7461121940999f6360a238b57a4d/users/523f6b84acdd4026b6574f5d69ee69a0/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.583094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898f7461121940999f6360a238b57a4d, user_id=523f6b84acdd4026b6574f5d69ee69a0, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.589988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.608953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:28.620837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1015/2036] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:28 2022] PUT /identity/v3/projects/898f7461121940999f6360a238b57a4d/users/523f6b84acdd4026b6574f5d69ee69a0/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:28.623921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.654670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1022/2037] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:28.655632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55c7c9-2c64-4df3-907f-b62f485dda54 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.675795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55c7c9-2c64-4df3-907f-b62f485dda54 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.736474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10643253-8cc4-4a81-ad1b-2fbf6107e648 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.747073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.747798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.748093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.748365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.749037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.751995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.757590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.758352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.758958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.759234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.760314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.764995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.783730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1023/2038] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:28.784987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.793659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1016/2039] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:28 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:28.795038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.798403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872f02-7837-473a-92fc-604f8b700fa0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.862020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:28.902992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8605-1832-46a6-8ca1-230aeec43b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.903849 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8605-1832-46a6-8ca1-230aeec43b36 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.904138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8605-1832-46a6-8ca1-230aeec43b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.904454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61d15c-ec0f-477a-bb7a-155c7447455b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:28.904587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8605-1832-46a6-8ca1-230aeec43b36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.905549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a8605-1832-46a6-8ca1-230aeec43b36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.910678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a8605-1832-46a6-8ca1-230aeec43b36 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.914838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:28.917932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.918752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.920159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] PATH_INFO: `/v3/projects/898f7461121940999f6360a238b57a4d/users/523f6b84acdd4026b6574f5d69ee69a0/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.922639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=898f7461121940999f6360a238b57a4d, user_id=523f6b84acdd4026b6574f5d69ee69a0, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:28.927270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:28.947674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1017/2040] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:28 2022] PUT /identity/v3/projects/898f7461121940999f6360a238b57a4d/users/523f6b84acdd4026b6574f5d69ee69a0/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:28.949916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:28.962282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1024/2041] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:28 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:28.963706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53e2b1-7966-4e14-997b-3ef42a786c68 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:28.964127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53e2b1-7966-4e14-997b-3ef42a786c68 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:28.964521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53e2b1-7966-4e14-997b-3ef42a786c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:28.994946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7c53e2b1-7966-4e14-997b-3ef42a786c68 None None] MFA Rules not processed for user `523f6b84acdd4026b6574f5d69ee69a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:28.996386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c53e2b1-7966-4e14-997b-3ef42a786c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:29.033363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce8d6-4605-4c6c-87d0-292d2a05f83c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.059033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:29.087838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1025/2042] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:29.088902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53e2b1-7966-4e14-997b-3ef42a786c68 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.109342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.110039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.110383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.110724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.111427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.113493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.138368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1018/2043] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:28 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:29.140843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.142827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:29.166121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53e2b1-7966-4e14-997b-3ef42a786c68 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.175374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.176335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.176620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.176952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.177668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.180418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.206927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1026/2044] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:29 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:29.207891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.216600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087c628-a6bc-4466-9dbc-93124590e067 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.220590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.220838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fff9a-f269-4a1d-8012-88b297ae4831 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.221324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.221475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.221749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.223510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:29.227107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.227874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.228139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.228524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.229061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.230697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.246356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1027/2045] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:29.246356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.258793 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:29.259343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a107-2ec3-4ffc-8667-0d4abbe5d924 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.264814 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.265431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.265777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.265983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.266634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.268276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.279773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1028/2046] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:29.280754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.290820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99e27c-4319-4f39-b55e-a8cf33655d2d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.296145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.296678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.296958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.297313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.297870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.299546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.308688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.310180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.313950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1029/2047] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:29.313950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.320904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1019/2048] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:29.326647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184972-b3b6-4ee3-8d32-597de280a65e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.328688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.332328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.333069 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.333429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.333778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] PATH_INFO: `/v3/projects/73900dded75b48868225b8342ce80265/users/143f9afae0cc4318b0fb4b309d964663/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.342152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578838bf-37b0-403a-9274-e42885726617 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.347728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1c348-276c-4974-a413-46b66e6c666d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.348514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c348-276c-4974-a413-46b66e6c666d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.348940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c348-276c-4974-a413-46b66e6c666d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.349292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c348-276c-4974-a413-46b66e6c666d admin admin] PATH_INFO: `/v3/projects/d4d649e0b0434716b8d4aaf4ea171a8e/users/2d2b7b4cb4a34964a4d6127a1ed3ac98/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.353202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73900dded75b48868225b8342ce80265, user_id=143f9afae0cc4318b0fb4b309d964663, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.355964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.366790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1c348-276c-4974-a413-46b66e6c666d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d649e0b0434716b8d4aaf4ea171a8e, user_id=2d2b7b4cb4a34964a4d6127a1ed3ac98, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.370639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1c348-276c-4974-a413-46b66e6c666d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.380836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1030/2049] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:29 2022] PUT /identity/v3/projects/73900dded75b48868225b8342ce80265/users/143f9afae0cc4318b0fb4b309d964663/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:29.386171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.395549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1020/2050] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:29 2022] PUT /identity/v3/projects/d4d649e0b0434716b8d4aaf4ea171a8e/users/2d2b7b4cb4a34964a4d6127a1ed3ac98/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:29.401147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.401435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.401712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.428729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:29.430437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:29.494184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac235-6c6f-456f-8699-520c293ba3c3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.532849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1021/2051] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:29.538034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:29.548616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.552658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:29.585883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.586735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.587013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.587287 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.590280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.592002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.609038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1031/2052] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:29.613478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.625733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef002-126c-42e9-88ad-aca470a0de16 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.632150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23142a3-e572-4433-9ad1-b75ecb38a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.632869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23142a3-e572-4433-9ad1-b75ecb38a4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.633155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23142a3-e572-4433-9ad1-b75ecb38a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.633435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23142a3-e572-4433-9ad1-b75ecb38a4f5 admin admin] PATH_INFO: `/v3/projects/73900dded75b48868225b8342ce80265/users/143f9afae0cc4318b0fb4b309d964663/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.635511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23142a3-e572-4433-9ad1-b75ecb38a4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73900dded75b48868225b8342ce80265, user_id=143f9afae0cc4318b0fb4b309d964663, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.638654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23142a3-e572-4433-9ad1-b75ecb38a4f5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.651224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cbc86-be2e-4acd-b531-3b8f1b1bb2c6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.657603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1032/2053] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:29 2022] PUT /identity/v3/projects/73900dded75b48868225b8342ce80265/users/143f9afae0cc4318b0fb4b309d964663/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:29.683778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de19d-9d41-4f6e-adf7-bb6429e14bc6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.684246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de19d-9d41-4f6e-adf7-bb6429e14bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.686418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de19d-9d41-4f6e-adf7-bb6429e14bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.761301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8b6de19d-9d41-4f6e-adf7-bb6429e14bc6 None None] MFA Rules not processed for user `143f9afae0cc4318b0fb4b309d964663`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:29.762537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b6de19d-9d41-4f6e-adf7-bb6429e14bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:29.769360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e06ba5-abb5-43ce-a341-9fefabc20ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.770551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e06ba5-abb5-43ce-a341-9fefabc20ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.770844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e06ba5-abb5-43ce-a341-9fefabc20ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.771277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e06ba5-abb5-43ce-a341-9fefabc20ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:29.808945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:29.891345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1033/2054] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:29.902474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e06ba5-abb5-43ce-a341-9fefabc20ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:29.904211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e06ba5-abb5-43ce-a341-9fefabc20ed3 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:29.904359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6de19d-9d41-4f6e-adf7-bb6429e14bc6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:29.922271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1022/2055] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:29.987433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6de19d-9d41-4f6e-adf7-bb6429e14bc6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:29.995808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:29.996659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:29.997003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:29.997343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:30.118706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:30.132514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:30.134108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:30.160093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1034/2056] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:30.955617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b98fcc-fe60-4a13-b7f4-132f54581191 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:30.956643 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b98fcc-fe60-4a13-b7f4-132f54581191 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:30.957157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b98fcc-fe60-4a13-b7f4-132f54581191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:30.999641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-47b98fcc-fe60-4a13-b7f4-132f54581191 None None] MFA Rules not processed for user `2d2b7b4cb4a34964a4d6127a1ed3ac98`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:31.008553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47b98fcc-fe60-4a13-b7f4-132f54581191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:31.045840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:31.047371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:31.066222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812f901-6e63-4d22-ab23-f164443906e6 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:31.076526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64766e6-641c-4fda-886b-a09be6984bd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:31.079254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64766e6-641c-4fda-886b-a09be6984bd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:31.079904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64766e6-641c-4fda-886b-a09be6984bd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:31.081308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64766e6-641c-4fda-886b-a09be6984bd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:31.086637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f64766e6-641c-4fda-886b-a09be6984bd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:31.139466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1023/2057] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:31.150647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b98fcc-fe60-4a13-b7f4-132f54581191 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:31.168391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b98fcc-fe60-4a13-b7f4-132f54581191 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:31.177265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:31.178237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:31.178775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:31.180229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:31.193481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:31.195602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:31.209175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1024/2058] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:31.214789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64766e6-641c-4fda-886b-a09be6984bd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:31.220772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64766e6-641c-4fda-886b-a09be6984bd7 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:31.245623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1035/2059] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:32.283054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:32.292254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0191-56bc-4504-9630-2290f69c7616 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:32.302718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:32.303623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:32.304663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:32.305325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:32.408354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:32.408354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:32.417933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1025/2060] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:32.670462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13048f2b-e3eb-44f9-9ee4-75b57a0560aa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:32.670968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13048f2b-e3eb-44f9-9ee4-75b57a0560aa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:32.670968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13048f2b-e3eb-44f9-9ee4-75b57a0560aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:32.689538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-13048f2b-e3eb-44f9-9ee4-75b57a0560aa None None] MFA Rules not processed for user `2d2b7b4cb4a34964a4d6127a1ed3ac98`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:32.690724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13048f2b-e3eb-44f9-9ee4-75b57a0560aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:32.698254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1036/2061] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:35.005737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:35.015051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad2151-11ab-4823-940d-5a31664fbe51 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:35.018655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d155c-a9b0-4e51-9cce-142c063ac947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:35.019253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d155c-a9b0-4e51-9cce-142c063ac947 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:35.019534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d155c-a9b0-4e51-9cce-142c063ac947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:35.019836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d155c-a9b0-4e51-9cce-142c063ac947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:35.029073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d155c-a9b0-4e51-9cce-142c063ac947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:35.030525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d155c-a9b0-4e51-9cce-142c063ac947 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:35.040452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1026/2062] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:36.979914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13048f2b-e3eb-44f9-9ee4-75b57a0560aa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:36.995319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13048f2b-e3eb-44f9-9ee4-75b57a0560aa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:37.002530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:37.003354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:37.003766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:37.004218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] PATH_INFO: `/v3/users/fc7d3e371f4f4a6182aab23747ad6a89` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:37.006237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7d3e371f4f4a6182aab23747ad6a89)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:37.009231 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:37.085661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7d3e371f4f4a6182aab23747ad6a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.105581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7d3e371f4f4a6182aab23747ad6a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.133630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7d3e371f4f4a6182aab23747ad6a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.134500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7d3e371f4f4a6182aab23747ad6a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.146513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7d3e371f4f4a6182aab23747ad6a89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.155728 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1037/2063] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:36 2022] DELETE /identity/v3/users/fc7d3e371f4f4a6182aab23747ad6a89 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:37.166739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:37.172794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:37.173068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:37.184297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:37.197962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:37.199177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:37.290922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1027/2064] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:37.307806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:37.309484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:37.420258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ea2ab-3100-411a-a6e8-ab735f87e689 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:37.425868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:37.426720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:37.427148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:37.427630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:37.438655 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:37.442455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:37.445405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:37.459279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1038/2065] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:37.499182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:37.501241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:37.516544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269cef4-5cb4-4728-be21-414a8372b39b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:37.522065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:37.523232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:37.523503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:37.523797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] PATH_INFO: `/v3/users/7bb9cf9d423f4fc3b2805bba0db064ee` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:37.526191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bb9cf9d423f4fc3b2805bba0db064ee)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:37.528272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:37.583495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bb9cf9d423f4fc3b2805bba0db064ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.584177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bb9cf9d423f4fc3b2805bba0db064ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.596944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bb9cf9d423f4fc3b2805bba0db064ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.614486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bb9cf9d423f4fc3b2805bba0db064ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.615297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bb9cf9d423f4fc3b2805bba0db064ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:37.630798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1028/2066] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:37 2022] DELETE /identity/v3/users/7bb9cf9d423f4fc3b2805bba0db064ee => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:37.841706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:37.854797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f81c1-f595-4099-a4c5-3cddb2080fd7 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:37.941447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:37.942369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:37.942798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:37.943278 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] PATH_INFO: `/v3/projects/079028929bb4478c8c3c2b474e74f109` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:37.944716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079028929bb4478c8c3c2b474e74f109)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:37.946816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:38.062375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079028929bb4478c8c3c2b474e74f109', 'request_id': 'req-ac25fd1e-708b-4849-b890-9e0ee4c51537'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.074084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079028929bb4478c8c3c2b474e74f109', 'request_id': 'req-ac25fd1e-708b-4849-b890-9e0ee4c51537'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.087325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079028929bb4478c8c3c2b474e74f109', 'request_id': 'req-ac25fd1e-708b-4849-b890-9e0ee4c51537'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.101436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] The token cache is being invalidate because project 079028929bb4478c8c3c2b474e74f109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:38.101731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.102396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.104374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1039/2067] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:37 2022] DELETE /identity/v3/projects/079028929bb4478c8c3c2b474e74f109 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:38.108668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:38.110500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:38.124602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94050d-78bb-4c7f-bea5-845aa7c8835f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:38.188133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:38.190437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:38.212617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:38.215023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:38.215855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:38.216653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] PATH_INFO: `/v3/users/9d67b1e32ef34deba8791de44bb35708` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:38.219256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d67b1e32ef34deba8791de44bb35708)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:38.222030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:38.246242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:38.276403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25fd1e-708b-4849-b890-9e0ee4c51537 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:38.280768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d67b1e32ef34deba8791de44bb35708'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.281474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d67b1e32ef34deba8791de44bb35708'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.281844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:38.282513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:38.282794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:38.283067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] PATH_INFO: `/v3/projects/6a927d4407b14bfa91f2279f18276e54` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:38.284220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a927d4407b14bfa91f2279f18276e54)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:38.286144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:38.293990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d67b1e32ef34deba8791de44bb35708'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.313043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d67b1e32ef34deba8791de44bb35708'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.313778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d67b1e32ef34deba8791de44bb35708'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.330554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1029/2068] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:38 2022] DELETE /identity/v3/users/9d67b1e32ef34deba8791de44bb35708 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:38.338190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:38.357425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a927d4407b14bfa91f2279f18276e54', 'request_id': 'req-64e8544b-8f58-4ca0-a61c-6f84f851d271'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.364319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a927d4407b14bfa91f2279f18276e54', 'request_id': 'req-64e8544b-8f58-4ca0-a61c-6f84f851d271'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.374778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a927d4407b14bfa91f2279f18276e54', 'request_id': 'req-64e8544b-8f58-4ca0-a61c-6f84f851d271'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.389883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] The token cache is being invalidate because project 6a927d4407b14bfa91f2279f18276e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:38.390169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.390844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.395130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1040/2069] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:38 2022] DELETE /identity/v3/projects/6a927d4407b14bfa91f2279f18276e54 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:38.397292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:38.400061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:38.489157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872b4fd-7d08-42d0-b7aa-5ba46d3c4153 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:38.628155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:38.628864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:38.629217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:38.629498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] PATH_INFO: `/v3/users/6d187d7a78ef4c73b6df2b1b50ea6b2b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:38.631307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d187d7a78ef4c73b6df2b1b50ea6b2b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:38.633863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:38.696810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:38.699784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d187d7a78ef4c73b6df2b1b50ea6b2b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.700910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d187d7a78ef4c73b6df2b1b50ea6b2b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.722967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d187d7a78ef4c73b6df2b1b50ea6b2b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.742712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d187d7a78ef4c73b6df2b1b50ea6b2b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.743507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9a84af90-57e3-4a05-8bdc-fb0178a40a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d187d7a78ef4c73b6df2b1b50ea6b2b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:38.760833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1030/2070] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:38 2022] DELETE /identity/v3/users/6d187d7a78ef4c73b6df2b1b50ea6b2b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:38.923831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:38.931904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:38.968919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8544b-8f58-4ca0-a61c-6f84f851d271 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.053615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.054820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.055225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.055552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] PATH_INFO: `/v3/projects/fcaa9492fa2849e1bc7ef707cb4d0219` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.056900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcaa9492fa2849e1bc7ef707cb4d0219)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.059536 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.145258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcaa9492fa2849e1bc7ef707cb4d0219', 'request_id': 'req-ea91f251-c269-4a5c-8118-02f10a907ab2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.161341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcaa9492fa2849e1bc7ef707cb4d0219', 'request_id': 'req-ea91f251-c269-4a5c-8118-02f10a907ab2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.182276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcaa9492fa2849e1bc7ef707cb4d0219', 'request_id': 'req-ea91f251-c269-4a5c-8118-02f10a907ab2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.206820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] The token cache is being invalidate because project fcaa9492fa2849e1bc7ef707cb4d0219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:39.207372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.208374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea91f251-c269-4a5c-8118-02f10a907ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.216497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051e12b-0359-4fe3-9ed2-98d0f6261008 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.217163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051e12b-0359-4fe3-9ed2-98d0f6261008 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.217341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051e12b-0359-4fe3-9ed2-98d0f6261008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.218674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1041/2071] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:38 2022] DELETE /identity/v3/projects/fcaa9492fa2849e1bc7ef707cb4d0219 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:39.291082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b051e12b-0359-4fe3-9ed2-98d0f6261008 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:39.293271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b051e12b-0359-4fe3-9ed2-98d0f6261008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:39.370369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1031/2072] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:39.378295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de149c-6e44-4edb-976b-ac27d5b16e1b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.378295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de149c-6e44-4edb-976b-ac27d5b16e1b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.378589 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de149c-6e44-4edb-976b-ac27d5b16e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.406594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-06de149c-6e44-4edb-976b-ac27d5b16e1b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:39.408062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06de149c-6e44-4edb-976b-ac27d5b16e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:39.428695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1042/2073] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:39.430466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051e12b-0359-4fe3-9ed2-98d0f6261008 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.441679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051e12b-0359-4fe3-9ed2-98d0f6261008 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.446529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.447140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.447415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.447682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.448505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:39.448899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.450585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.463462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1032/2074] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:12:39.468325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de149c-6e44-4edb-976b-ac27d5b16e1b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.478629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de149c-6e44-4edb-976b-ac27d5b16e1b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.483885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.484506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.484818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.485086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.485712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.487947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.519298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1043/2075] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:39 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:39.524317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.535293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee33f-2554-4f1f-9408-4f3df6259b7f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.597881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.598494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.598770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.599037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.599690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.602233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.625220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1033/2076] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:39 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:39.629185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.639620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1ddd2-5c61-4daf-89a4-0d3a18f5f4f1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.644484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.645087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.645781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.646061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.646835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.648940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.662598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1044/2077] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:39.664652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.675596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df774f-046e-4c58-9398-811806ecd528 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.681604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.682365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.682720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.683063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.683801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.685925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.698840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1034/2078] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:39.703111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.723182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2d510-948b-4378-9062-387879ee7eba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.738965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.743974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:39.751499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a67497-a7d8-4a68-a064-14e91631324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.752238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67497-a7d8-4a68-a064-14e91631324b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.752519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67497-a7d8-4a68-a064-14e91631324b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.752804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a67497-a7d8-4a68-a064-14e91631324b admin admin] PATH_INFO: `/v3/projects/97856657a7ed4a408620a5523bfe34a0/users/f5203090840341b7a4b6c8abf8dcd37f/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.755479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:39.783231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44715e1a-36c3-42c5-92cf-c02db9349ca5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:39.788688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a67497-a7d8-4a68-a064-14e91631324b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97856657a7ed4a408620a5523bfe34a0, user_id=f5203090840341b7a4b6c8abf8dcd37f, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.789339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:39.790603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.791588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.792123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] PATH_INFO: `/v3/projects/73802156e51542eeb539235cee007650` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.792323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a67497-a7d8-4a68-a064-14e91631324b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.793843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73802156e51542eeb539235cee007650)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:39.796329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:39.815948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1045/2079] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:39 2022] PUT /identity/v3/projects/97856657a7ed4a408620a5523bfe34a0/users/f5203090840341b7a4b6c8abf8dcd37f/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:39.826962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:39.827245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:39.827521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:39.858191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:39.881229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] MFA Rules not processed for user `f5203090840341b7a4b6c8abf8dcd37f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:39.883423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:39.901022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73802156e51542eeb539235cee007650', 'request_id': 'req-40e58c9f-7925-4438-8806-6ca6159a4b37'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.911947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73802156e51542eeb539235cee007650', 'request_id': 'req-40e58c9f-7925-4438-8806-6ca6159a4b37'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.928266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73802156e51542eeb539235cee007650', 'request_id': 'req-40e58c9f-7925-4438-8806-6ca6159a4b37'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.959464 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] The token cache is being invalidate because project 73802156e51542eeb539235cee007650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:39.959852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.961053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:39.966094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1035/2080] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:39 2022] DELETE /identity/v3/projects/73802156e51542eeb539235cee007650 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:39.966959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:39.968409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:39.998681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1046/2081] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:40.009419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:40.011185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:40.084061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e58c9f-7925-4438-8806-6ca6159a4b37 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:40.089402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:40.090403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:40.091365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:40.091885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] PATH_INFO: `/v3/users/523f6b84acdd4026b6574f5d69ee69a0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:40.093405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=523f6b84acdd4026b6574f5d69ee69a0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:40.095678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:40.130078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8639a-f578-44f8-9794-e062b8986b50 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:40.139859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:40.140558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:40.140957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:40.141813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:40.160921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:40.165141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:40.183462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523f6b84acdd4026b6574f5d69ee69a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.184167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523f6b84acdd4026b6574f5d69ee69a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.191915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1047/2082] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:40.196263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523f6b84acdd4026b6574f5d69ee69a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.228401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523f6b84acdd4026b6574f5d69ee69a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.230453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ade0fd3f-1857-47b6-ad40-cb2102b8a255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523f6b84acdd4026b6574f5d69ee69a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.247218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1036/2083] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:39 2022] DELETE /identity/v3/users/523f6b84acdd4026b6574f5d69ee69a0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:40.387041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:40.388531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:40.413057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae9ae-bb90-4af9-bc05-5afdb6119d07 service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:40.519865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:40.530954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:40.531919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:40.532747 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:40.533057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] PATH_INFO: `/v3/users/8e212eb6f9c24bc1af28795e36494e90` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:40.534223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e212eb6f9c24bc1af28795e36494e90)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:40.536739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:40.608342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e212eb6f9c24bc1af28795e36494e90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.619013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e212eb6f9c24bc1af28795e36494e90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.637209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e212eb6f9c24bc1af28795e36494e90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.638110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e212eb6f9c24bc1af28795e36494e90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.647572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e212eb6f9c24bc1af28795e36494e90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:40.659200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1048/2084] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:40 2022] DELETE /identity/v3/users/8e212eb6f9c24bc1af28795e36494e90 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:40.660995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:40.661436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:40.661710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:40.685773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:40.686987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:40.727222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:40.729004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:40.745826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:40.808394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1037/2085] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:40.820080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:40.822255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:40.881130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3182ef-6a33-4036-b939-ce999991c9c3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:40.886497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:40.887204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:40.887552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:40.887727 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] PATH_INFO: `/v3/projects/898f7461121940999f6360a238b57a4d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:40.892952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=898f7461121940999f6360a238b57a4d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:40.894968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:40.944401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63f152-63d5-43df-8f3b-d816a39066d2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:40.954112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee222a64-8523-429a-b832-75b32e9d1cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:40.954747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222a64-8523-429a-b832-75b32e9d1cfe service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:40.955032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222a64-8523-429a-b832-75b32e9d1cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:40.955311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222a64-8523-429a-b832-75b32e9d1cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:40.981373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee222a64-8523-429a-b832-75b32e9d1cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:40.983356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee222a64-8523-429a-b832-75b32e9d1cfe service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.020789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '898f7461121940999f6360a238b57a4d', 'request_id': 'req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.028198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '898f7461121940999f6360a238b57a4d', 'request_id': 'req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.034077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1038/2086] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:41.044168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '898f7461121940999f6360a238b57a4d', 'request_id': 'req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.070226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] The token cache is being invalidate because project 898f7461121940999f6360a238b57a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:41.071073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.071574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d2b33b1-2fa4-4fa6-b13f-2f030104dd7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.073942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1049/2087] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:40 2022] DELETE /identity/v3/projects/898f7461121940999f6360a238b57a4d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:41.078641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cfc55-1b2e-44fb-b37a-ece186ebbfd3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.078925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cfc55-1b2e-44fb-b37a-ece186ebbfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.079197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cfc55-1b2e-44fb-b37a-ece186ebbfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.125972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8b1cfc55-1b2e-44fb-b37a-ece186ebbfd3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:41.127598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8b1cfc55-1b2e-44fb-b37a-ece186ebbfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:41.205262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1039/2088] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:41.213154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac61bb-4348-4df3-845b-b85f7c71aa76 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.214125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac61bb-4348-4df3-845b-b85f7c71aa76 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.214591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac61bb-4348-4df3-845b-b85f7c71aa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.237776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:41.274301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3bac61bb-4348-4df3-845b-b85f7c71aa76 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:41.276223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3bac61bb-4348-4df3-845b-b85f7c71aa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:41.289498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1050/2089] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:41.294214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cfc55-1b2e-44fb-b37a-ece186ebbfd3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.306012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cfc55-1b2e-44fb-b37a-ece186ebbfd3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.311019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.311713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.312103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.312438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.313185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:41.313490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.315931 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.330623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1040/2090] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:12:41.337541 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac61bb-4348-4df3-845b-b85f7c71aa76 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.357624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac61bb-4348-4df3-845b-b85f7c71aa76 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.373522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.377155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.377599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.378025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.380329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.384157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.419206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.422943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:41.433180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1051/2091] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:41 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:41.436482 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.448921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65859d5-2f5a-470a-a09c-a53f86f87a34 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.571682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aacaaf-7022-4cce-ab39-a672cb63f840 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.606015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.606784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.607165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.607539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.608430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.617592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.621792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.622462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.622748 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.623026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] PATH_INFO: `/v3/projects/fd0ac8be80ab485a8724cc74b3d50f8f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.624247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0ac8be80ab485a8724cc74b3d50f8f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.626774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.630693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:41.667961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1052/2092] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:41 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:41.672113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.692947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d266370-0ab1-45f3-b673-fe216c3baba5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.714036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0ac8be80ab485a8724cc74b3d50f8f', 'request_id': 'req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.721030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0ac8be80ab485a8724cc74b3d50f8f', 'request_id': 'req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.752132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0ac8be80ab485a8724cc74b3d50f8f', 'request_id': 'req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.777554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] The token cache is being invalidate because project fd0ac8be80ab485a8724cc74b3d50f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:41.777907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.778902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.779063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.780946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.780946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.780946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.781178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1041/2093] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:41 2022] DELETE /identity/v3/projects/fd0ac8be80ab485a8724cc74b3d50f8f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:41.784515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.785012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.786321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.786669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:41.802671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893e30-0163-4f07-a50f-dc1ca0b51b9a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.805940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1053/2094] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:41.809768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.809943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.810478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.810761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.811037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] PATH_INFO: `/v3/users/143f9afae0cc4318b0fb4b309d964663` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.811685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:41.812215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] RBAC: Authorizing `identity:delete_user(user_id=143f9afae0cc4318b0fb4b309d964663)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.814325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.833755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9b2c6-1848-46ad-a145-974dd6217150 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.841833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.842465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.843636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.843636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.843943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.845532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.858830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1054/2095] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:41.859901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.861582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:41.873674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '143f9afae0cc4318b0fb4b309d964663'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.874355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '143f9afae0cc4318b0fb4b309d964663'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.875417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325055c-f882-4d5b-9109-20093e128034 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:41.880422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:41.881058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:41.881339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:41.881658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] PATH_INFO: `/v3/users/0354d80eb898418fae8acf8baf4894b1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:41.882810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0354d80eb898418fae8acf8baf4894b1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:41.883795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '143f9afae0cc4318b0fb4b309d964663'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.884831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:41.902042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '143f9afae0cc4318b0fb4b309d964663'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.902791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '143f9afae0cc4318b0fb4b309d964663'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.917839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1042/2096] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:41 2022] DELETE /identity/v3/users/143f9afae0cc4318b0fb4b309d964663 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:41.919248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:41.921326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:41.976600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:41.981104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0354d80eb898418fae8acf8baf4894b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:41.996492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0354d80eb898418fae8acf8baf4894b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.005510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287a73c-fa2b-4b11-8ba3-ebb3d147930c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.013341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0354d80eb898418fae8acf8baf4894b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.014258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.015072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.015383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.015655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] PATH_INFO: `/v3/projects/9ead4033c51d4658a5406c045da1c26e/users/453fa4c58743419bb29e5f01d9f78041/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.016780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0354d80eb898418fae8acf8baf4894b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.026399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:42.039030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ead4033c51d4658a5406c045da1c26e, user_id=453fa4c58743419bb29e5f01d9f78041, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.042063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57e6cf8-4a35-400a-b9f0-6829e66d6990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0354d80eb898418fae8acf8baf4894b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.042438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.051307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1055/2097] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:41 2022] DELETE /identity/v3/users/0354d80eb898418fae8acf8baf4894b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:42.060156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1043/2098] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:41 2022] PUT /identity/v3/projects/9ead4033c51d4658a5406c045da1c26e/users/453fa4c58743419bb29e5f01d9f78041/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:42.067041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.067933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.068238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.096972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:42.098204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.187809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1056/2099] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:42.205179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.207789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.244899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.247513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.263967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09727a-30e1-4735-ab97-41ee39664063 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.290714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b474ff-e653-4753-b3f1-b4c09f807669 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.290981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.291861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.292153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.292439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] PATH_INFO: `/v3/projects/73900dded75b48868225b8342ce80265` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.294593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73900dded75b48868225b8342ce80265)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.295652 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.296285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.296552 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.296823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.297150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.308942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.312732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.326204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1044/2100] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:42.327070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.328395 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.341581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c6e6a-7db0-4a17-8f11-e912dbc86fbf service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.353599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.354245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.354524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.354796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] PATH_INFO: `/v3/projects/2002558f3d814b1b892ca2d22ff25f05` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.355989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2002558f3d814b1b892ca2d22ff25f05)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.357890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.384491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73900dded75b48868225b8342ce80265', 'request_id': 'req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.392147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73900dded75b48868225b8342ce80265', 'request_id': 'req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.405319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73900dded75b48868225b8342ce80265', 'request_id': 'req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.420726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2002558f3d814b1b892ca2d22ff25f05', 'request_id': 'req-3bf06490-4106-4bea-a315-b33c368dc9e7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.421511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] The token cache is being invalidate because project 73900dded75b48868225b8342ce80265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:42.422307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.423206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f1e71d1-04f3-4839-b9bc-d6a402f59fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.425159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1057/2101] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:42 2022] DELETE /identity/v3/projects/73900dded75b48868225b8342ce80265 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:42.427673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2002558f3d814b1b892ca2d22ff25f05', 'request_id': 'req-3bf06490-4106-4bea-a315-b33c368dc9e7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.431386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09231968-785f-4128-9206-460883ad81fc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.431666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09231968-785f-4128-9206-460883ad81fc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.431967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09231968-785f-4128-9206-460883ad81fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.442117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2002558f3d814b1b892ca2d22ff25f05', 'request_id': 'req-3bf06490-4106-4bea-a315-b33c368dc9e7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.454682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-09231968-785f-4128-9206-460883ad81fc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:42.455715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] The token cache is being invalidate because project 2002558f3d814b1b892ca2d22ff25f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:42.456046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.456899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3bf06490-4106-4bea-a315-b33c368dc9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:42.459343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1045/2102] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:42 2022] DELETE /identity/v3/projects/2002558f3d814b1b892ca2d22ff25f05 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:42.460184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-09231968-785f-4128-9206-460883ad81fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.464968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1609d16-df1f-4365-8d0f-4919eb9a00da None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.465266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1609d16-df1f-4365-8d0f-4919eb9a00da None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.465607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1609d16-df1f-4365-8d0f-4919eb9a00da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.490434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c1609d16-df1f-4365-8d0f-4919eb9a00da None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:42.492356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c1609d16-df1f-4365-8d0f-4919eb9a00da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.548741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:42.593565 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1058/2103] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:42.595950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f11a6-f0c6-4cbb-aee4-30d11678c4e2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.596272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f11a6-f0c6-4cbb-aee4-30d11678c4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.596495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f11a6-f0c6-4cbb-aee4-30d11678c4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.619458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1046/2104] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:42.619912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aa4f11a6-f0c6-4cbb-aee4-30d11678c4e2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:42.620404 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa4f11a6-f0c6-4cbb-aee4-30d11678c4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.622489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390134f-ccc3-4d81-a711-f7dcd071ac13 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.623002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390134f-ccc3-4d81-a711-f7dcd071ac13 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.623424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390134f-ccc3-4d81-a711-f7dcd071ac13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.629885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1059/2105] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:42.634837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f11a6-f0c6-4cbb-aee4-30d11678c4e2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.646277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f11a6-f0c6-4cbb-aee4-30d11678c4e2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.648104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9390134f-ccc3-4d81-a711-f7dcd071ac13 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:42.649226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9390134f-ccc3-4d81-a711-f7dcd071ac13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:42.651588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.652501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.652717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.653055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.653774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:42.654139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.655864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.659994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1047/2106] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:42.664471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390134f-ccc3-4d81-a711-f7dcd071ac13 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.676100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1060/2107] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:12:42.677945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390134f-ccc3-4d81-a711-f7dcd071ac13 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.680531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.683271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.684032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.684384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.684727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.685447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:42.685842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.687798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.690312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:42.692970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dea77-07cc-4d72-844f-986fa5882a35 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.698881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.699622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.700029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.701005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.701978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.705370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.711783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1048/2108] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:12:42.715490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.748788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02985bae-e66a-47be-ac34-3baf175a406e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.754396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1061/2109] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:42.760622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.835232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.835958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.836240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.836514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.837146 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.841242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.848257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c179-6fca-404e-bc28-f09c97143df5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.857134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.858129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.858624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.859153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:42.860042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:42.862420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:42.890617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1049/2110] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:42.895249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.896341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1062/2111] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:42.899976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:42.912378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895db92-5395-4f09-b55c-15793b03e3fb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.995925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef3b5-ab38-437b-b8d2-70a44b2eec97 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:42.996425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:42.997421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:42.997825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:42.998179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.001271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.002592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.003224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.003499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.004096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.005152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.007548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.012554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.022851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1063/2112] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:43.029023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.039422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1050/2113] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:42 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:43.045486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.047023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838df942-5d23-487e-9c7a-85b01adcf3bd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.053279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.054908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.055200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.055540 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.056290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.056957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9774b-7c19-4f50-982b-7611ee07e68d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.058208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.063174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.063842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.064318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.064686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.065615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.067638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.070982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1064/2114] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:43.074891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.079261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1051/2115] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:43.082943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.087074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471169e-eba0-42e0-a6a6-d2a736a8b74a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.092717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735cefd-8d55-44a8-9460-a2474189a4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.093466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735cefd-8d55-44a8-9460-a2474189a4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.093647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735cefd-8d55-44a8-9460-a2474189a4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.093914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735cefd-8d55-44a8-9460-a2474189a4e7 admin admin] PATH_INFO: `/v3/projects/42ba4b19dc0746fcbfdf2509a10b3783/users/ec0386fa14ef4e83bba3184c4941726f/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.094173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5797abf-0273-433d-85c5-73eec1aafa2a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.099524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.099949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.100878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.101262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.102017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.103887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.113165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735cefd-8d55-44a8-9460-a2474189a4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ba4b19dc0746fcbfdf2509a10b3783, user_id=ec0386fa14ef4e83bba3184c4941726f, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.116792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1052/2116] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:43.117274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735cefd-8d55-44a8-9460-a2474189a4e7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.122072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:43.126968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.138511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1065/2117] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:43 2022] PUT /identity/v3/projects/42ba4b19dc0746fcbfdf2509a10b3783/users/ec0386fa14ef4e83bba3184c4941726f/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:43.140927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7d00-83a0-493a-97ba-44632548050e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.151713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244ac6-e591-4b12-a897-40b2762d658d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.154184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244ac6-e591-4b12-a897-40b2762d658d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.154648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244ac6-e591-4b12-a897-40b2762d658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.189299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ff244ac6-e591-4b12-a897-40b2762d658d None None] MFA Rules not processed for user `ec0386fa14ef4e83bba3184c4941726f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:43.190594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff244ac6-e591-4b12-a897-40b2762d658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:43.207534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:43.215972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243bd82-d5ba-41d8-bdc6-7ca1528794b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.216828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243bd82-d5ba-41d8-bdc6-7ca1528794b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.217577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243bd82-d5ba-41d8-bdc6-7ca1528794b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.218438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243bd82-d5ba-41d8-bdc6-7ca1528794b2 admin admin] PATH_INFO: `/v3/projects/80cfec738cab491ba413e062b5a00813/users/e17a4e664b2c4ac9a9fea65287faad95/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.235179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243bd82-d5ba-41d8-bdc6-7ca1528794b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cfec738cab491ba413e062b5a00813, user_id=e17a4e664b2c4ac9a9fea65287faad95, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.238321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243bd82-d5ba-41d8-bdc6-7ca1528794b2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.263286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1053/2118] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:43 2022] PUT /identity/v3/projects/80cfec738cab491ba413e062b5a00813/users/e17a4e664b2c4ac9a9fea65287faad95/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:43.269250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b2db-2583-4653-8546-31b79a2b1c74 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.269959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b2db-2583-4653-8546-31b79a2b1c74 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.270243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b2db-2583-4653-8546-31b79a2b1c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.287421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1066/2119] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:12:43.293596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff244ac6-e591-4b12-a897-40b2762d658d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.298721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bd67b2db-2583-4653-8546-31b79a2b1c74 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:43.299980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bd67b2db-2583-4653-8546-31b79a2b1c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:43.306365 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff244ac6-e591-4b12-a897-40b2762d658d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.386102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1054/2120] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:43.387645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d78c-33a5-4069-9eed-89ad52861a52 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.388140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d78c-33a5-4069-9eed-89ad52861a52 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.388584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d78c-33a5-4069-9eed-89ad52861a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.416273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.416890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.417170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.417443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.418073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.420245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.430576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5b61d78c-33a5-4069-9eed-89ad52861a52 None None] MFA Rules not processed for user `453fa4c58743419bb29e5f01d9f78041`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:43.432963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5b61d78c-33a5-4069-9eed-89ad52861a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:43.455182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1067/2121] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:43 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:43.455182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.462009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:43.518718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1055/2122] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:43.519432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61d78c-33a5-4069-9eed-89ad52861a52 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.531023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61d78c-33a5-4069-9eed-89ad52861a52 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.534676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806587b8-7174-4957-b1b9-2ba515a3d0ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.539167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.539810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.540096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.540390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.588458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:43.617479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.618089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.618366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.618739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.619261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.621223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.630733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:43.637847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.639603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.644559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1056/2123] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:43 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:43.645608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.648128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:43.654429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1068/2124] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:43.654905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.664726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a11f2e-3287-48c4-a8ab-acc78cf64332 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.670416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.671116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.671397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.671668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.672354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.673985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.697916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1069/2125] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:43.698908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.716118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ef6f1-92ef-4945-95be-f10859e379dc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.720473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601206cf-50b7-4cd7-a563-73e5f84c29a2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.722867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.723550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.723871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.724152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] PATH_INFO: `/v3/projects/658de114966e4af8bbcdf126d80b7530/users/bedf18774cbb4745b146f901d9fa7dd8/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.724675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.727021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.727021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.727237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.745916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658de114966e4af8bbcdf126d80b7530, user_id=bedf18774cbb4745b146f901d9fa7dd8, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.749110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.764427 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1070/2126] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:43 2022] PUT /identity/v3/projects/658de114966e4af8bbcdf126d80b7530/users/bedf18774cbb4745b146f901d9fa7dd8/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:43.768443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.799038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.800730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.813369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1057/2127] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:43.846506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb985a0e-d8b2-4a7f-a769-ad2a268967d7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.853834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9166d9f-f1de-4a70-aab0-b60d3cad25c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.854973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9166d9f-f1de-4a70-aab0-b60d3cad25c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.856128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9166d9f-f1de-4a70-aab0-b60d3cad25c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.856806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9166d9f-f1de-4a70-aab0-b60d3cad25c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.858010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9166d9f-f1de-4a70-aab0-b60d3cad25c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.860453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9166d9f-f1de-4a70-aab0-b60d3cad25c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.879065 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1071/2128] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:43.880727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:43.898184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6de92-786a-4f1b-ab6e-19768edb96bf service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:43.903510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:43.904230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.904509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.904791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] PATH_INFO: `/v3/projects/658de114966e4af8bbcdf126d80b7530/users/bedf18774cbb4745b146f901d9fa7dd8/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.907048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=658de114966e4af8bbcdf126d80b7530, user_id=bedf18774cbb4745b146f901d9fa7dd8, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:43.910076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:43.921797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1058/2129] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:43 2022] PUT /identity/v3/projects/658de114966e4af8bbcdf126d80b7530/users/bedf18774cbb4745b146f901d9fa7dd8/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:43.932908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e84021-6e11-4043-a6aa-819d3b6439d5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:43.933107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e84021-6e11-4043-a6aa-819d3b6439d5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:43.933477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e84021-6e11-4043-a6aa-819d3b6439d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:43.980789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-93e84021-6e11-4043-a6aa-819d3b6439d5 None None] MFA Rules not processed for user `bedf18774cbb4745b146f901d9fa7dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:43.983190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-93e84021-6e11-4043-a6aa-819d3b6439d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:44.085499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1072/2130] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:44.145475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:44.148088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:44.277691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0cc45-f257-41b4-bdb7-2566b881d06d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:44.289473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:44.292190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:44.292599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:44.292976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:44.432264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:44.450587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:44.452454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:44.464427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1059/2131] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:45.325155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8288eb-ff8a-40c6-a034-3e72ab106cef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:45.326181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8288eb-ff8a-40c6-a034-3e72ab106cef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:45.326474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8288eb-ff8a-40c6-a034-3e72ab106cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:45.344243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:45.354202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bb8288eb-ff8a-40c6-a034-3e72ab106cef None None] MFA Rules not processed for user `453fa4c58743419bb29e5f01d9f78041`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:45.356137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb8288eb-ff8a-40c6-a034-3e72ab106cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:45.426257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1073/2132] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:47.391359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:47.538054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87862bc7-412c-45a4-b298-216669eeaac7 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:47.549239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:47.550202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:47.550595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:47.550967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] PATH_INFO: `/v3/users/ec0386fa14ef4e83bba3184c4941726f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:47.554005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0386fa14ef4e83bba3184c4941726f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:47.557403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:47.673735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0386fa14ef4e83bba3184c4941726f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:47.674586 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0386fa14ef4e83bba3184c4941726f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:47.690088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0386fa14ef4e83bba3184c4941726f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:47.708356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0386fa14ef4e83bba3184c4941726f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:47.708560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0386fa14ef4e83bba3184c4941726f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:47.727553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1060/2133] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:47 2022] DELETE /identity/v3/users/ec0386fa14ef4e83bba3184c4941726f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:47.731955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:47.732813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:47.733092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:47.776083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:47.790612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:47.886081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1074/2134] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:47.902216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:47.911565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:47.994903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cac67-5a88-4f18-a24e-8205230903ea admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:47.998787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:47.999416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:47.999770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:48.000050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:48.010414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:48.011900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:48.024271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1061/2135] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:48.452813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:48.454272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:48.477178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde4989-ca26-4b1b-bbdc-76d08bcbbe08 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:48.485118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:48.485537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:48.485951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:48.486247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] PATH_INFO: `/v3/projects/42ba4b19dc0746fcbfdf2509a10b3783` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:48.487780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ba4b19dc0746fcbfdf2509a10b3783)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:48.489754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:48.607250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ba4b19dc0746fcbfdf2509a10b3783', 'request_id': 'req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.627648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ba4b19dc0746fcbfdf2509a10b3783', 'request_id': 'req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.641122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ba4b19dc0746fcbfdf2509a10b3783', 'request_id': 'req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.664390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] The token cache is being invalidate because project 42ba4b19dc0746fcbfdf2509a10b3783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:48.664695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.665544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.671830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1075/2136] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:48 2022] DELETE /identity/v3/projects/42ba4b19dc0746fcbfdf2509a10b3783 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:48.673751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:48.684594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51934433-53a7-47a4-aa62-caea671dc20a service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:48.689564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:48.773915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:48.774582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:48.775289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:48.777483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] PATH_INFO: `/v3/users/bedf18774cbb4745b146f901d9fa7dd8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:48.780309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bedf18774cbb4745b146f901d9fa7dd8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:48.785022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:48.863223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bedf18774cbb4745b146f901d9fa7dd8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.863977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bedf18774cbb4745b146f901d9fa7dd8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.873848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bedf18774cbb4745b146f901d9fa7dd8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.895472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bedf18774cbb4745b146f901d9fa7dd8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.896534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-936beb50-49ec-4f74-b9a0-280a995ce037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bedf18774cbb4745b146f901d9fa7dd8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:48.928039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1062/2137] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:48 2022] DELETE /identity/v3/users/bedf18774cbb4745b146f901d9fa7dd8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:49.495073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:49.500360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:49.559420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba528c-8969-4f87-b7f5-2ca0eb9d232e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:49.663931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:49.748993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:49.749639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:49.749918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:49.750187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] PATH_INFO: `/v3/projects/658de114966e4af8bbcdf126d80b7530` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:49.756844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658de114966e4af8bbcdf126d80b7530)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:49.759513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:49.828869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658de114966e4af8bbcdf126d80b7530', 'request_id': 'req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:49.845596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658de114966e4af8bbcdf126d80b7530', 'request_id': 'req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:49.863618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658de114966e4af8bbcdf126d80b7530', 'request_id': 'req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:49.880397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] The token cache is being invalidate because project 658de114966e4af8bbcdf126d80b7530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:12:49.880625 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:49.881292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cc9441cd-489a-4855-a18c-dd9a8a6b88c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:49.883789 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1076/2138] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:12:49 2022] DELETE /identity/v3/projects/658de114966e4af8bbcdf126d80b7530 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:49.888204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87623e4d-fe1f-41f2-85bf-72c46a74c4eb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:49.888691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87623e4d-fe1f-41f2-85bf-72c46a74c4eb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:49.888770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87623e4d-fe1f-41f2-85bf-72c46a74c4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:49.928391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-87623e4d-fe1f-41f2-85bf-72c46a74c4eb None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:49.929952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-87623e4d-fe1f-41f2-85bf-72c46a74c4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:49.989579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:49.991314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:49.991314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.069042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1063/2139] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:50.070450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] MFA Rules not processed for user `e17a4e664b2c4ac9a9fea65287faad95`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:50.071588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c58c-74a9-4467-a2af-7061f1ba2906 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.072138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c58c-74a9-4467-a2af-7061f1ba2906 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.072269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:50.072547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2c58c-74a9-4467-a2af-7061f1ba2906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.108363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-abb2c58c-74a9-4467-a2af-7061f1ba2906 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:50.109973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-abb2c58c-74a9-4467-a2af-7061f1ba2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:50.120808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1064/2140] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:50.125023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2c58c-74a9-4467-a2af-7061f1ba2906 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:50.143128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2c58c-74a9-4467-a2af-7061f1ba2906 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:50.154659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:50.155384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.155844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.156167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.156829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:12:50.157177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:50.158842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:50.173537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1065/2141] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:12:50.179039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:50.202925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487257b-68e5-4774-84e2-c1aad06ccddb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:50.210497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:50.211122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.211401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.211674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.212409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:50.214357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:50.221129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:50.230798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1077/2142] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:50.247520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:50.259269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:50.270323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1066/2143] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:50 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:50.276320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:50.371826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:50.402495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613b9e4-e1f3-45ce-8cd6-43c2c6e4ebc0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:50.414117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8f799-c85b-4091-8fd9-af88a490f400 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:50.415631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:50.418665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.419116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.419555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.421512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:50.424091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:50.462107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1067/2144] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:50 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:50.463229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:50.487819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c90d68-17ba-4fae-abf9-950c83d1f148 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:50.505580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:50.507655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.508098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.508392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.509083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:50.513729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:50.541798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1068/2145] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:50.543722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:50.549987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35976e8b-e0b9-4cb4-87f1-39f66b26275d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:50.550615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35976e8b-e0b9-4cb4-87f1-39f66b26275d service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.551060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35976e8b-e0b9-4cb4-87f1-39f66b26275d service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.551345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35976e8b-e0b9-4cb4-87f1-39f66b26275d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.587388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55721d-a27e-4349-97bf-69534b1aac20 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:50.602843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:50.603450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.603841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.604231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.605773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:50.611101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:50.629242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1069/2146] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:50.634633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:50.647734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1bc0-3134-4806-be8a-08ef5264b3a5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:50.660206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:50.660206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.660206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.660566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] PATH_INFO: `/v3/projects/574dada930f94111918b60a8d9136503/users/4dc435daadfe414a83bc16d860a61593/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.691679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35976e8b-e0b9-4cb4-87f1-39f66b26275d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:50.695228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35976e8b-e0b9-4cb4-87f1-39f66b26275d service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:50.697240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574dada930f94111918b60a8d9136503, user_id=4dc435daadfe414a83bc16d860a61593, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:50.702539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:50.719714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1078/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:50.733183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1070/2148] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:50 2022] PUT /identity/v3/projects/574dada930f94111918b60a8d9136503/users/4dc435daadfe414a83bc16d860a61593/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:50.746400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef641e-4a98-489a-bb01-c6f9f628b277 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:50.746660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef641e-4a98-489a-bb01-c6f9f628b277 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:50.748181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef641e-4a98-489a-bb01-c6f9f628b277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:50.786007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eeef641e-4a98-489a-bb01-c6f9f628b277 None None] MFA Rules not processed for user `4dc435daadfe414a83bc16d860a61593`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:50.788021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eeef641e-4a98-489a-bb01-c6f9f628b277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:50.981556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1079/2149] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:50.985670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:51.104223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6336c-454e-4137-b701-b0b45c753473 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:51.109986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:51.111149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:51.111149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:51.111310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:51.111949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:51.113881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:51.163239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1071/2150] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:50 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:51.165797 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef641e-4a98-489a-bb01-c6f9f628b277 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:51.184378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef641e-4a98-489a-bb01-c6f9f628b277 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:51.216836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:51.293873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:51.294822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:51.295501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:51.295825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:51.296475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:51.298416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:51.335800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1080/2151] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:51 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:12:51.339303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:51.352629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86446-14f5-4d23-a598-82c52f9df407 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:51.358802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:51.360346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:51.360690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:51.360969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:51.361857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:51.363632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:51.380481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1072/2152] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:51.382601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:51.395079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f86ca8-609c-4636-95ff-a7d22218d184 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:51.402065 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29058-c083-47a3-95eb-886e6d05b09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:51.402948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29058-c083-47a3-95eb-886e6d05b09f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:51.403507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29058-c083-47a3-95eb-886e6d05b09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:51.403983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29058-c083-47a3-95eb-886e6d05b09f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:51.404921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e29058-c083-47a3-95eb-886e6d05b09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:51.407101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e29058-c083-47a3-95eb-886e6d05b09f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:51.436616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1081/2153] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:12:51.441467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:51.450647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:51.458380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b03e85-91da-4586-a0d7-eca28e7f2e36 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:51.467941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:51.470215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:51.472880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:51.473591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] PATH_INFO: `/v3/projects/7e080529456a4ee39e56f7d59250afbd/users/37235a0f73224a42801d550bcb3fdece/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:51.504727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e080529456a4ee39e56f7d59250afbd, user_id=37235a0f73224a42801d550bcb3fdece, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:51.507935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:51.527646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1073/2154] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:51 2022] PUT /identity/v3/projects/7e080529456a4ee39e56f7d59250afbd/users/37235a0f73224a42801d550bcb3fdece/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:51.538579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbf598-9c55-41b0-a1d5-c1f819e7b2d7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:51.538869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbf598-9c55-41b0-a1d5-c1f819e7b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:51.539246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbf598-9c55-41b0-a1d5-c1f819e7b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:51.632830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-45fbf598-9c55-41b0-a1d5-c1f819e7b2d7 None None] MFA Rules not processed for user `37235a0f73224a42801d550bcb3fdece`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:51.635938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-45fbf598-9c55-41b0-a1d5-c1f819e7b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:51.749679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1082/2155] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:51.761884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:51.918951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4699f-cd37-48ed-9eb7-a53612df37d2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:51.927326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:51.928273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:51.928782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:51.929256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:51.930930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:51.934576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:51.984244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1074/2156] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:12:51 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:51.985851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbf598-9c55-41b0-a1d5-c1f819e7b2d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:52.020802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbf598-9c55-41b0-a1d5-c1f819e7b2d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:52.141596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:52.143223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.143805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.144809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.145788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:52.149832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:52.198472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1083/2157] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:12:51 2022] POST /identity/v3/users => generated 295 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:12:52.200244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:52.214358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7e1ea-695e-4eb5-9f3c-012c6f10cc38 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:52.220252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:52.221951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.222307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.222657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.223735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:52.226145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:52.248287 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1075/2158] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:52.249114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:52.260177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4503d9-85f4-4548-9819-477594e3c744 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:52.274743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:52.275645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.275978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.276294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] PATH_INFO: `/v3/projects/21d4b6ab2e654575a16fd0ca9fa7d74a/users/32253cd3006844ca81df6454ebc25c3b/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.305966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d4b6ab2e654575a16fd0ca9fa7d74a, user_id=32253cd3006844ca81df6454ebc25c3b, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:52.311000 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:52.316374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:52.339584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1084/2159] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:52 2022] PUT /identity/v3/projects/21d4b6ab2e654575a16fd0ca9fa7d74a/users/32253cd3006844ca81df6454ebc25c3b/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:52.343588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:52.361544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0837440-72a7-4509-9dd7-a571f7008058 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:52.458472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a533a6-7e16-40ef-bee9-76f40a00d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:52.459218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a533a6-7e16-40ef-bee9-76f40a00d489 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.459591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a533a6-7e16-40ef-bee9-76f40a00d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.460020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a533a6-7e16-40ef-bee9-76f40a00d489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.465241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a533a6-7e16-40ef-bee9-76f40a00d489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:52.466546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a533a6-7e16-40ef-bee9-76f40a00d489 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:52.478383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1076/2160] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:12:52.482222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:52.497710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b40a52-331d-4d07-9669-cafaa24aa40a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:52.503304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:52.503660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.503987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.504261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] PATH_INFO: `/v3/projects/21d4b6ab2e654575a16fd0ca9fa7d74a/users/32253cd3006844ca81df6454ebc25c3b/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.506502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d4b6ab2e654575a16fd0ca9fa7d74a, user_id=32253cd3006844ca81df6454ebc25c3b, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:52.510435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:52.528175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1085/2161] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:12:52 2022] PUT /identity/v3/projects/21d4b6ab2e654575a16fd0ca9fa7d74a/users/32253cd3006844ca81df6454ebc25c3b/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:12:52.539951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb394-5cd9-4993-ba22-894df511eaed None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.540248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb394-5cd9-4993-ba22-894df511eaed None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.540664 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb394-5cd9-4993-ba22-894df511eaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.604691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-581eb394-5cd9-4993-ba22-894df511eaed None None] MFA Rules not processed for user `32253cd3006844ca81df6454ebc25c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:52.606431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-581eb394-5cd9-4993-ba22-894df511eaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:52.657578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:52.715877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1077/2162] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:52.728136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:52.731248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:52.832708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693dab4b-460b-4d11-8a7f-ddfa292ad7af admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:52.837226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:52.837845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.838124 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.838943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.912422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9edd-bc29-4264-ad4e-717a2ec96893 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:52.913207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9edd-bc29-4264-ad4e-717a2ec96893 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:52.913479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9edd-bc29-4264-ad4e-717a2ec96893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:52.935356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-883a9edd-bc29-4264-ad4e-717a2ec96893 None None] MFA Rules not processed for user `e17a4e664b2c4ac9a9fea65287faad95`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:52.936169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:52.936925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-883a9edd-bc29-4264-ad4e-717a2ec96893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:52.938218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:52.964224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1086/2163] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:53.011534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1078/2164] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:54.609372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:54.610806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:54.622903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea88b2f-4906-457e-a450-8981017211d4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:54.626957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:54.627945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:54.628235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:54.628511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:54.702794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:54.704712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:54.730271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1087/2165] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:12:55.427000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a9edd-bc29-4264-ad4e-717a2ec96893 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:55.437438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a9edd-bc29-4264-ad4e-717a2ec96893 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:55.441008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f69be5-ec98-4435-9aac-ef4860a19db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:55.441607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f69be5-ec98-4435-9aac-ef4860a19db0 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:55.441890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f69be5-ec98-4435-9aac-ef4860a19db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:55.442162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f69be5-ec98-4435-9aac-ef4860a19db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:55.455951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f69be5-ec98-4435-9aac-ef4860a19db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:55.457394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f69be5-ec98-4435-9aac-ef4860a19db0 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:55.468313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1079/2166] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:12:59.602291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:59.648903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:12:59.680571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46bd53-85a1-4068-933e-624377ee7474 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:59.685465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:59.686176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:59.686515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:59.686850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] PATH_INFO: `/v3/users/4dc435daadfe414a83bc16d860a61593` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:59.688071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc435daadfe414a83bc16d860a61593)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:59.690042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:12:59.740785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc435daadfe414a83bc16d860a61593'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:59.748986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc435daadfe414a83bc16d860a61593'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:59.763127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc435daadfe414a83bc16d860a61593'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:59.764045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc435daadfe414a83bc16d860a61593'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:59.771022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc435daadfe414a83bc16d860a61593'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:12:59.778004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1088/2167] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:12:59 2022] DELETE /identity/v3/users/4dc435daadfe414a83bc16d860a61593 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:12:59.783788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:59.784630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:59.784630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:59.809994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:12:59.811273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:59.886125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1080/2168] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:12:59.896333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:12:59.898254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:12:59.964838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41c642-916a-475f-8912-94a48dfece7b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:12:59.969564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:12:59.970683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:12:59.971378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:12:59.971943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:12:59.986604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:12:59.989104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:00.005501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1089/2169] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:13:00.399896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:00.401557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:00.415486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95554682-aa04-481f-b341-d1cd85b4bf04 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:00.425366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:00.426021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:00.426301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:00.426572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] PATH_INFO: `/v3/projects/574dada930f94111918b60a8d9136503` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:00.428842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=574dada930f94111918b60a8d9136503)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:00.430811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:00.489159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574dada930f94111918b60a8d9136503', 'request_id': 'req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.499331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574dada930f94111918b60a8d9136503', 'request_id': 'req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.508950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:00.518377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574dada930f94111918b60a8d9136503', 'request_id': 'req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.538060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] The token cache is being invalidate because project 574dada930f94111918b60a8d9136503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:13:00.538358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.539032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.543889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1081/2170] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:13:00 2022] DELETE /identity/v3/projects/574dada930f94111918b60a8d9136503 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:00.545233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:00.561961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f60a0c-ecf8-433f-97b1-93da46d71c2b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:00.619134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:00.628697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:00.629654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:00.630020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:00.630398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] PATH_INFO: `/v3/users/37235a0f73224a42801d550bcb3fdece` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:00.632619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37235a0f73224a42801d550bcb3fdece)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:00.635580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:00.693453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37235a0f73224a42801d550bcb3fdece'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.702542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37235a0f73224a42801d550bcb3fdece'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.718359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37235a0f73224a42801d550bcb3fdece'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.719152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37235a0f73224a42801d550bcb3fdece'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.728091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37235a0f73224a42801d550bcb3fdece'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:00.735029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1090/2171] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:13:00 2022] DELETE /identity/v3/users/37235a0f73224a42801d550bcb3fdece => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:01.032444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:01.034154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:01.076226 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabfb31-5b3b-4e97-8c0b-96ab864a57ee admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:01.155885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:01.156602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:01.156952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:01.157316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] PATH_INFO: `/v3/projects/7e080529456a4ee39e56f7d59250afbd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:01.158493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e080529456a4ee39e56f7d59250afbd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:01.160487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:01.207823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e080529456a4ee39e56f7d59250afbd', 'request_id': 'req-d0b042e1-815a-4104-a1d6-096dc5373609'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.215143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e080529456a4ee39e56f7d59250afbd', 'request_id': 'req-d0b042e1-815a-4104-a1d6-096dc5373609'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.224191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e080529456a4ee39e56f7d59250afbd', 'request_id': 'req-d0b042e1-815a-4104-a1d6-096dc5373609'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.236346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] The token cache is being invalidate because project 7e080529456a4ee39e56f7d59250afbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:13:01.236645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.237771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.239264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1082/2172] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:13:01 2022] DELETE /identity/v3/projects/7e080529456a4ee39e56f7d59250afbd => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:01.243363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:01.244940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:01.257980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bdadf-ed04-4347-b7d6-f99c1b8ab1e4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:01.317381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:01.318085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:01.318396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:01.318723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] PATH_INFO: `/v3/users/32253cd3006844ca81df6454ebc25c3b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:01.319946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32253cd3006844ca81df6454ebc25c3b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:01.322066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:01.370418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32253cd3006844ca81df6454ebc25c3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.376866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32253cd3006844ca81df6454ebc25c3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.388704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32253cd3006844ca81df6454ebc25c3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.389353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32253cd3006844ca81df6454ebc25c3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.393665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:01.398330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32253cd3006844ca81df6454ebc25c3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.405271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1091/2173] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:13:01 2022] DELETE /identity/v3/users/32253cd3006844ca81df6454ebc25c3b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:01.685438 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:01.686988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:01.700829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b042e1-815a-4104-a1d6-096dc5373609 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:01.759563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:01.760219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:01.760559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:01.760834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] PATH_INFO: `/v3/projects/21d4b6ab2e654575a16fd0ca9fa7d74a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:01.761932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d4b6ab2e654575a16fd0ca9fa7d74a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:01.763892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:01.788887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:01.817147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d4b6ab2e654575a16fd0ca9fa7d74a', 'request_id': 'req-b29089f8-d334-4dca-8ac2-ea16b0702540'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.823252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d4b6ab2e654575a16fd0ca9fa7d74a', 'request_id': 'req-b29089f8-d334-4dca-8ac2-ea16b0702540'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.832602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d4b6ab2e654575a16fd0ca9fa7d74a', 'request_id': 'req-b29089f8-d334-4dca-8ac2-ea16b0702540'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.838011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:01.839414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:01.844581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] The token cache is being invalidate because project 21d4b6ab2e654575a16fd0ca9fa7d74a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:13:01.844972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.845673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b29089f8-d334-4dca-8ac2-ea16b0702540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.847250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1083/2174] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:13:01 2022] DELETE /identity/v3/projects/21d4b6ab2e654575a16fd0ca9fa7d74a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:13:01.851960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e67b2f-ba7f-49d5-9d06-4fe7b1fdae40 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:01.852247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e67b2f-ba7f-49d5-9d06-4fe7b1fdae40 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:01.852349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbb724-980e-47ca-ba91-ad8df37ed968 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:01.852520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e67b2f-ba7f-49d5-9d06-4fe7b1fdae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:01.873700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-01e67b2f-ba7f-49d5-9d06-4fe7b1fdae40 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:01.874915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-01e67b2f-ba7f-49d5-9d06-4fe7b1fdae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:01.916643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:01.917304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:01.917586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:01.917863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] PATH_INFO: `/v3/users/f5203090840341b7a4b6c8abf8dcd37f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:01.919258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5203090840341b7a4b6c8abf8dcd37f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:01.921230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:01.939407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1084/2175] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:13:01.947648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e02b6-adc8-4eb3-b3b7-ec8b3ec0ecff None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:01.948038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e02b6-adc8-4eb3-b3b7-ec8b3ec0ecff None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:01.948315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e02b6-adc8-4eb3-b3b7-ec8b3ec0ecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:01.971024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5203090840341b7a4b6c8abf8dcd37f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.971850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2d7e02b6-adc8-4eb3-b3b7-ec8b3ec0ecff None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:01.973115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2d7e02b6-adc8-4eb3-b3b7-ec8b3ec0ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:01.979719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5203090840341b7a4b6c8abf8dcd37f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.982136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1085/2176] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:13:01.986339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e02b6-adc8-4eb3-b3b7-ec8b3ec0ecff None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:01.992410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5203090840341b7a4b6c8abf8dcd37f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.993071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5203090840341b7a4b6c8abf8dcd37f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:01.995735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e02b6-adc8-4eb3-b3b7-ec8b3ec0ecff None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.000461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.000708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7fbc642e-ba33-409a-89ee-2cad58fd6b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5203090840341b7a4b6c8abf8dcd37f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:02.001039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.001323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.001588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.002224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:13:02.002522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.004142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.007454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1092/2177] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:13:01 2022] DELETE /identity/v3/users/f5203090840341b7a4b6c8abf8dcd37f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:13:02.013153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.013438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.013717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.023356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1086/2178] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:13:02.024329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.025975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:02.033166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:02.034350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:02.040672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4caaba-baaa-4450-999c-b08e0e9fed65 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.105494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.106063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.106165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1093/2179] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:02.106483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.106760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.107537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.110144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.117998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.119624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:02.145144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:02.185595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1087/2180] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:13:02 2022] POST /identity/v3/projects => generated 369 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:13:02.186647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.216898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:02.221596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa5528-5415-49f5-becc-4f683ad49382 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.306363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47015084-5832-45b0-9d76-061b8509f7a8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.308761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.310476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.310770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.311073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.319111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.320223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.320502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.320776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.321424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.327008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.333493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.335164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.366084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1088/2181] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:13:02 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:13:02.369896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1094/2182] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:13:02.372968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.395709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66560ad8-c6b7-436f-b8dc-5bd96e189aec admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.423855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.423855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.423855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.423855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.424352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.424352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.448448 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1089/2183] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:13:02.452451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.479238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13e92-4cc6-4de8-95bf-45c540acc346 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.484241 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d4be-8576-4852-9cd9-0c4804e361f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.484851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d4be-8576-4852-9cd9-0c4804e361f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.485192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d4be-8576-4852-9cd9-0c4804e361f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.485527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d4be-8576-4852-9cd9-0c4804e361f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.486169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d4be-8576-4852-9cd9-0c4804e361f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.487804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d4be-8576-4852-9cd9-0c4804e361f8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.507862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1095/2184] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:13:02.510776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.526765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcecde-7bc3-4949-a668-fb6921142d0b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.533099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.533631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.533919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.534184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] PATH_INFO: `/v3/projects/d1f67b17cc4842c69c4c9d6b73991405/users/72d3f543c59340edbc8ad1f3d95161b9/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.559049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f67b17cc4842c69c4c9d6b73991405, user_id=72d3f543c59340edbc8ad1f3d95161b9, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.562579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.593282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1090/2185] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:13:02 2022] PUT /identity/v3/projects/d1f67b17cc4842c69c4c9d6b73991405/users/72d3f543c59340edbc8ad1f3d95161b9/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:02.606088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.607081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.607479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.680322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] MFA Rules not processed for user `72d3f543c59340edbc8ad1f3d95161b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:02.680322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:02.762290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1096/2186] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:02.766183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.769368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:02.775405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:02.778638 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:02.854886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d88e4-dc06-4c10-868a-b464933ddff2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.863615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.864415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.864628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.864915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] PATH_INFO: `/v3/projects/97856657a7ed4a408620a5523bfe34a0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.866756 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97856657a7ed4a408620a5523bfe34a0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.869390 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.875081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077467b-99e8-47e7-8a09-a9a31c6b85a1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:02.879155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d593dbc-976b-4be7-9df8-434bfa5af503 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:02.879811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d593dbc-976b-4be7-9df8-434bfa5af503 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.880615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d593dbc-976b-4be7-9df8-434bfa5af503 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.881465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d593dbc-976b-4be7-9df8-434bfa5af503 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:02.890701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:02.896200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d593dbc-976b-4be7-9df8-434bfa5af503 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:02.897994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d593dbc-976b-4be7-9df8-434bfa5af503 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:02.909055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1097/2187] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:13:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:13:02.937901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97856657a7ed4a408620a5523bfe34a0', 'request_id': 'req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:02.946337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97856657a7ed4a408620a5523bfe34a0', 'request_id': 'req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:02.957208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97856657a7ed4a408620a5523bfe34a0', 'request_id': 'req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:02.966376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:02.970927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] The token cache is being invalidate because project 97856657a7ed4a408620a5523bfe34a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:13:02.971976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:02.973686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a42666e-7011-4f1a-b2aa-ebe2254b23c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:02.975796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1091/2188] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:13:02 2022] DELETE /identity/v3/projects/97856657a7ed4a408620a5523bfe34a0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:02.980512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280808-4d33-4ebc-820c-f0e37e7fa8c1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:02.980879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280808-4d33-4ebc-820c-f0e37e7fa8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:02.981230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37280808-4d33-4ebc-820c-f0e37e7fa8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.010000 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-37280808-4d33-4ebc-820c-f0e37e7fa8c1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:03.011475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-37280808-4d33-4ebc-820c-f0e37e7fa8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:03.082180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1098/2189] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:03.090989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d7158-7654-4b8b-a17b-04097b9e327f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.091685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d7158-7654-4b8b-a17b-04097b9e327f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.091685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d7158-7654-4b8b-a17b-04097b9e327f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.114624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c57d7158-7654-4b8b-a17b-04097b9e327f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:03.116484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c57d7158-7654-4b8b-a17b-04097b9e327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:03.125704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1092/2190] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:03.130607 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37280808-4d33-4ebc-820c-f0e37e7fa8c1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.144867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37280808-4d33-4ebc-820c-f0e37e7fa8c1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.151708 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.152783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.153123 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.153345 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.154038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:13:03.154347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.155997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.170823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1099/2191] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:13:03.175630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d7158-7654-4b8b-a17b-04097b9e327f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.187682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d7158-7654-4b8b-a17b-04097b9e327f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.193357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.193967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.194250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.194551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.195188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.197137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.244086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1093/2192] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:13:03 2022] POST /identity/v3/projects => generated 361 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:13:03.249025 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.259356 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af52955-882b-4536-8c54-7b962767fb25 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.338525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.339148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.339536 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.340051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.342027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.344301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.378101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1100/2193] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:13:03 2022] POST /identity/v3/users => generated 281 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:13:03.389604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.405747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8c27d-2029-4dcf-b6d5-ff0309938a1a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.411836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.412995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.413280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.413553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.414356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.417245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.435067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1094/2194] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:13:03.436483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.460495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a0ec2-ba85-47d6-95ac-557ab612007f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.465510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efe479-986e-4989-a50f-75367ba2e085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.466265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efe479-986e-4989-a50f-75367ba2e085 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.466879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efe479-986e-4989-a50f-75367ba2e085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.467224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efe479-986e-4989-a50f-75367ba2e085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.467988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efe479-986e-4989-a50f-75367ba2e085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.469709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efe479-986e-4989-a50f-75367ba2e085 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.488261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1101/2195] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:13:03.492244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.503253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce32ecd-37a7-46fb-900a-b9ee930c84e2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.511848 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.512686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.512972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.513253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] PATH_INFO: `/v3/projects/cc769959b3b544ac8a3906c40039ffb1/users/6db33704390a4603843d25405a30ebd2/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.530461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc769959b3b544ac8a3906c40039ffb1, user_id=6db33704390a4603843d25405a30ebd2, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.533217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.550819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1095/2196] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:13:03 2022] PUT /identity/v3/projects/cc769959b3b544ac8a3906c40039ffb1/users/6db33704390a4603843d25405a30ebd2/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:03.564739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7059e8-23f8-4227-83c5-8fe563aa8062 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.565041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7059e8-23f8-4227-83c5-8fe563aa8062 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.565372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7059e8-23f8-4227-83c5-8fe563aa8062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.597218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6e7059e8-23f8-4227-83c5-8fe563aa8062 None None] MFA Rules not processed for user `6db33704390a4603843d25405a30ebd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:03.598575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6e7059e8-23f8-4227-83c5-8fe563aa8062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:03.659289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:03.695885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1102/2197] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:03.738159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.740295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:03.812359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fdd4a-09c6-4c50-936d-63c17746b770 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.816382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c592632-3e76-4aa9-81e8-de32e9e5866f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.816987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c592632-3e76-4aa9-81e8-de32e9e5866f service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.817393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c592632-3e76-4aa9-81e8-de32e9e5866f service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.817672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c592632-3e76-4aa9-81e8-de32e9e5866f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.828393 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c592632-3e76-4aa9-81e8-de32e9e5866f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.830538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c592632-3e76-4aa9-81e8-de32e9e5866f service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.844676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1096/2198] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:13:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:13:03.860286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7059e8-23f8-4227-83c5-8fe563aa8062 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:03.938288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7059e8-23f8-4227-83c5-8fe563aa8062 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:03.942917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:03.944117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:03.944400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:03.944675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] PATH_INFO: `/v3/users/6db33704390a4603843d25405a30ebd2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:03.945818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db33704390a4603843d25405a30ebd2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:03.947719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:03.995583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db33704390a4603843d25405a30ebd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.002776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db33704390a4603843d25405a30ebd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.016367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db33704390a4603843d25405a30ebd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.017091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db33704390a4603843d25405a30ebd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.024689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db33704390a4603843d25405a30ebd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.030917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1103/2199] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:13:03 2022] DELETE /identity/v3/users/6db33704390a4603843d25405a30ebd2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:04.036295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.036589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.036866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.058960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:04.060219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:04.125197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1097/2200] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:04.135932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:04.137367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:04.198103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41badc6f-77e7-42d7-ac1e-1c16bbab2ab6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:04.202381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47689ba8-510d-4640-b6ad-053302856eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:04.203164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47689ba8-510d-4640-b6ad-053302856eea service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.203449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47689ba8-510d-4640-b6ad-053302856eea service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.203719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47689ba8-510d-4640-b6ad-053302856eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.216057 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47689ba8-510d-4640-b6ad-053302856eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:04.217515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47689ba8-510d-4640-b6ad-053302856eea service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:04.227697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1104/2201] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:13:04.514863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:04.516428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:04.531023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ce8b-548c-4a60-a2df-46cb92d88ffd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:04.536162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:04.537562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.537985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.538462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] PATH_INFO: `/v3/projects/cc769959b3b544ac8a3906c40039ffb1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.540174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc769959b3b544ac8a3906c40039ffb1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:04.544143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:04.584049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:04.595789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc769959b3b544ac8a3906c40039ffb1', 'request_id': 'req-b09fda89-e5bf-4609-829a-ed172c71c97f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.601710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc769959b3b544ac8a3906c40039ffb1', 'request_id': 'req-b09fda89-e5bf-4609-829a-ed172c71c97f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.609362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc769959b3b544ac8a3906c40039ffb1', 'request_id': 'req-b09fda89-e5bf-4609-829a-ed172c71c97f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.621039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] The token cache is being invalidate because project cc769959b3b544ac8a3906c40039ffb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:13:04.621331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.621960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b09fda89-e5bf-4609-829a-ed172c71c97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:04.623468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1098/2202] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:13:04 2022] DELETE /identity/v3/projects/cc769959b3b544ac8a3906c40039ffb1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:04.629047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b76ef-3c5a-42ea-8cc2-00a2f8b525e6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.629332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b76ef-3c5a-42ea-8cc2-00a2f8b525e6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.629623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b76ef-3c5a-42ea-8cc2-00a2f8b525e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.650147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-735b76ef-3c5a-42ea-8cc2-00a2f8b525e6 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:04.651398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-735b76ef-3c5a-42ea-8cc2-00a2f8b525e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:04.656800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:04.716829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1105/2203] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:04.726136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60608406-cfb8-4c96-b68b-919e274ca6ea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.726777 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60608406-cfb8-4c96-b68b-919e274ca6ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.727165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60608406-cfb8-4c96-b68b-919e274ca6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.756134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-60608406-cfb8-4c96-b68b-919e274ca6ea None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:04.756134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-60608406-cfb8-4c96-b68b-919e274ca6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:04.764771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1099/2204] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:13:04.770314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b76ef-3c5a-42ea-8cc2-00a2f8b525e6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:04.780050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b76ef-3c5a-42ea-8cc2-00a2f8b525e6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:04.784956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:04.786226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.786578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.787531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.788301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:13:04.788722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:04.790393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:04.809530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1106/2205] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:13:04.810845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60608406-cfb8-4c96-b68b-919e274ca6ea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:04.822209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60608406-cfb8-4c96-b68b-919e274ca6ea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:04.827774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:04.828774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.828774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.828951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.829531 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:04.831434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:04.866466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1100/2206] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:13:04 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:13:04.870975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:04.881705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6efe3c-e8a6-4e5c-a9e6-defb5593b93f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:04.962651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:04.964019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:04.964685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:04.965031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:04.965856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:04.968230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:04.995783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1107/2207] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:13:04 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:13:04.997295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:05.008585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c550a-2f9c-4895-a5c1-ecfdf38a252c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:05.017289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:05.018016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:05.018363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:05.018700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:05.019435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:05.021130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:05.031928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1101/2208] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:13:05.035898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:05.047151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e41b7a-5cbc-4dac-a362-71867eea97ce admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:05.051884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20029668-920e-4760-9008-a9471f1add47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:05.052508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029668-920e-4760-9008-a9471f1add47 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:05.052794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029668-920e-4760-9008-a9471f1add47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:05.053068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029668-920e-4760-9008-a9471f1add47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:05.053706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20029668-920e-4760-9008-a9471f1add47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:05.055418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20029668-920e-4760-9008-a9471f1add47 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:05.070276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1108/2209] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:13:05.075122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:05.087573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ecc66-d2ef-4a9a-b37f-d008d59f5683 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:05.092767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:05.093437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:05.093750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:05.094587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] PATH_INFO: `/v3/projects/cc6946068f2c40c8b717a4a3df3d95ce/users/a0a4d826e18648a39377479f8713353a/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:05.110358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6946068f2c40c8b717a4a3df3d95ce, user_id=a0a4d826e18648a39377479f8713353a, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:05.116352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:05.140252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1102/2210] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:13:05 2022] PUT /identity/v3/projects/cc6946068f2c40c8b717a4a3df3d95ce/users/a0a4d826e18648a39377479f8713353a/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:13:05.154052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c92e8d-c61c-477f-950c-03de304121ca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:05.154339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c92e8d-c61c-477f-950c-03de304121ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:05.154637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c92e8d-c61c-477f-950c-03de304121ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:05.185036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-10c92e8d-c61c-477f-950c-03de304121ca None None] MFA Rules not processed for user `a0a4d826e18648a39377479f8713353a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:05.186257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-10c92e8d-c61c-477f-950c-03de304121ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:05.251442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1109/2211] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:08.059467 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:08.061217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:08.124928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5572c8-f1e1-4dde-aef3-2b905a00a836 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:08.128515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c4aed-09f9-4def-8ec3-72db3ebde48a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:08.129125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4aed-09f9-4def-8ec3-72db3ebde48a service glance] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:08.129410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4aed-09f9-4def-8ec3-72db3ebde48a service glance] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:08.129687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c4aed-09f9-4def-8ec3-72db3ebde48a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:08.138204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c4aed-09f9-4def-8ec3-72db3ebde48a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:08.139964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c4aed-09f9-4def-8ec3-72db3ebde48a service glance] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:08.150106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1103/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:13:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:13:08.276818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45e4b-1c1e-4fa0-8c1d-0d1f897e54a3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:08.277294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45e4b-1c1e-4fa0-8c1d-0d1f897e54a3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:08.277371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45e4b-1c1e-4fa0-8c1d-0d1f897e54a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:08.292407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:08.300892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cec45e4b-1c1e-4fa0-8c1d-0d1f897e54a3 None None] MFA Rules not processed for user `453fa4c58743419bb29e5f01d9f78041`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:08.303212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cec45e4b-1c1e-4fa0-8c1d-0d1f897e54a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:08.374652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1110/2213] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:18.145338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:18.364432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=22 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:38.220964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf896b5a-cf6d-4a03-8b0e-6a6127d388ec None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:38.221537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf896b5a-cf6d-4a03-8b0e-6a6127d388ec None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:38.221537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf896b5a-cf6d-4a03-8b0e-6a6127d388ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:38.243134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cf896b5a-cf6d-4a03-8b0e-6a6127d388ec None None] MFA Rules not processed for user `f8f44df824414e5ebc8cfd2126e9d66f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:38.244560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cf896b5a-cf6d-4a03-8b0e-6a6127d388ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:38.304710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1104/2214] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:48.296077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=25 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:57.095946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec45e4b-1c1e-4fa0-8c1d-0d1f897e54a3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:57.184468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec45e4b-1c1e-4fa0-8c1d-0d1f897e54a3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:57.190931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:57.192890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:57.193389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:57.194041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] PATH_INFO: `/v3/users/a0a4d826e18648a39377479f8713353a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:57.195021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a4d826e18648a39377479f8713353a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:57.197440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:57.258937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a4d826e18648a39377479f8713353a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.268767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a4d826e18648a39377479f8713353a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.284225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a4d826e18648a39377479f8713353a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.284969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a4d826e18648a39377479f8713353a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.292556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a4d826e18648a39377479f8713353a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.306700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1111/2215] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:13:57 2022] DELETE /identity/v3/users/a0a4d826e18648a39377479f8713353a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:57.306866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:57.306866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:57.306866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:57.323239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:57.324555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:57.388422 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1105/2216] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:57.398909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:57.400386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:57.461557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58b586-bf43-4d1d-9db4-5421413e9c86 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:57.465403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c698e-7094-44ce-af49-f9beb904c395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:57.466108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c698e-7094-44ce-af49-f9beb904c395 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:57.466501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c698e-7094-44ce-af49-f9beb904c395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:57.466868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c698e-7094-44ce-af49-f9beb904c395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:57.476776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c698e-7094-44ce-af49-f9beb904c395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:57.478261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c698e-7094-44ce-af49-f9beb904c395 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:57.488019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1112/2217] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:13:57.767708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:57.769224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:57.807391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea4fa2-96fa-47f9-b5e6-f4643ad503a2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:57.813167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:57.813808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:57.814270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:57.814547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] PATH_INFO: `/v3/projects/cc6946068f2c40c8b717a4a3df3d95ce` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:57.815880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6946068f2c40c8b717a4a3df3d95ce)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:57.817788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:57.869055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6946068f2c40c8b717a4a3df3d95ce', 'request_id': 'req-7999ec86-63bb-4b33-b63a-cadc4ab7730e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.875428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6946068f2c40c8b717a4a3df3d95ce', 'request_id': 'req-7999ec86-63bb-4b33-b63a-cadc4ab7730e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.884647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6946068f2c40c8b717a4a3df3d95ce', 'request_id': 'req-7999ec86-63bb-4b33-b63a-cadc4ab7730e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.897588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] The token cache is being invalidate because project cc6946068f2c40c8b717a4a3df3d95ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:13:57.897967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.898675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7999ec86-63bb-4b33-b63a-cadc4ab7730e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:13:57.900339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1106/2218] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:13:57 2022] DELETE /identity/v3/projects/cc6946068f2c40c8b717a4a3df3d95ce => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:57.905744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd3e27-cd1e-4a37-9b69-45f606d793de None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:57.906010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd3e27-cd1e-4a37-9b69-45f606d793de None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:57.906289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd3e27-cd1e-4a37-9b69-45f606d793de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:57.940517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5cfd3e27-cd1e-4a37-9b69-45f606d793de None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:57.941658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5cfd3e27-cd1e-4a37-9b69-45f606d793de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:57.960587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:58.013157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1113/2219] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:58.020671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509da81f-3bfc-4769-8be5-4a1a8d9e44f0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.020961 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509da81f-3bfc-4769-8be5-4a1a8d9e44f0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.021237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509da81f-3bfc-4769-8be5-4a1a8d9e44f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.041133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-509da81f-3bfc-4769-8be5-4a1a8d9e44f0 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:58.042343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-509da81f-3bfc-4769-8be5-4a1a8d9e44f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:58.050736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1107/2220] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:58.056591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd3e27-cd1e-4a37-9b69-45f606d793de None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:58.068926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd3e27-cd1e-4a37-9b69-45f606d793de None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:58.073615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:58.074380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.074726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.075055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.075793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:13:58.076109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:58.077994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:58.091423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1114/2221] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:13:58.094864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509da81f-3bfc-4769-8be5-4a1a8d9e44f0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:58.104563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509da81f-3bfc-4769-8be5-4a1a8d9e44f0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:58.109960 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:58.110708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.111061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.111408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.112168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:58.114378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:58.143468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1108/2222] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:13:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:13:58.147407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:58.157220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38233f5-90a3-48cb-9fec-de24e0417d6c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:58.213194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:58.213724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.214003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.214408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.215044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:58.216966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:58.237156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1115/2223] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:13:58 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:13:58.241755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:58.251982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a23b9e-e16c-4922-bada-144d4abb4f40 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:58.257189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:58.257788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.258091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.258358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.258996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:58.260655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:58.270974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1109/2224] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:13:58.274863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:58.284838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ab8e-e604-46c5-b7de-7abd5f27dc04 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:58.289408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f85365-ae52-4046-a418-3660dbd5aa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:58.289998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f85365-ae52-4046-a418-3660dbd5aa3b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.290279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f85365-ae52-4046-a418-3660dbd5aa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.290568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f85365-ae52-4046-a418-3660dbd5aa3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.291184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f85365-ae52-4046-a418-3660dbd5aa3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:58.292834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f85365-ae52-4046-a418-3660dbd5aa3b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:58.308973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1116/2225] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:13:58.310349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:58.320891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc128cd2-a31d-4ae7-8ceb-9bb5a6e47185 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:58.325500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:58.326187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.326468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.326746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] PATH_INFO: `/v3/projects/3d234f0dfc3547e19a0900b5e916c747/users/dc8b7254fd7644418442a0a7619e8188/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.342759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d234f0dfc3547e19a0900b5e916c747, user_id=dc8b7254fd7644418442a0a7619e8188, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:58.345444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:58.360396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1110/2226] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:13:58 2022] PUT /identity/v3/projects/3d234f0dfc3547e19a0900b5e916c747/users/dc8b7254fd7644418442a0a7619e8188/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:13:58.372085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3abbe-9f9e-4d86-9781-00ec99136d4b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.372374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3abbe-9f9e-4d86-9781-00ec99136d4b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.372646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3abbe-9f9e-4d86-9781-00ec99136d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.401059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6af3abbe-9f9e-4d86-9781-00ec99136d4b None None] MFA Rules not processed for user `dc8b7254fd7644418442a0a7619e8188`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:58.402364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6af3abbe-9f9e-4d86-9781-00ec99136d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:58.482820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1117/2227] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:58.493918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:58.495620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:58.516804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:58.553620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe566700-0154-4f0f-a18c-0f0b322ecd54 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:58.557261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:58.557856 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:58.558135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:58.558407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:58.566380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:58.567817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:58.576477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1111/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:13:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:13:59.706852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd44dc8-ecdb-4ce6-a5d0-02c7f3c6c5c2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:59.707348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd44dc8-ecdb-4ce6-a5d0-02c7f3c6c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:59.707652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd44dc8-ecdb-4ce6-a5d0-02c7f3c6c5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:59.729024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0cd44dc8-ecdb-4ce6-a5d0-02c7f3c6c5c2 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:13:59.730236 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0cd44dc8-ecdb-4ce6-a5d0-02c7f3c6c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:59.744239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:13:59.794837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1118/2229] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:13:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:13:59.838071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:13:59.839559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:13:59.910326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893038e-68d9-499c-9317-90af1f37b7c1 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:13:59.916179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:13:59.916644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:13:59.916928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:13:59.917209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:13:59.926170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:13:59.928385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:13:59.951035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1112/2230] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:13:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:04.270820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:04.271980 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:04.272080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:04.302757 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:04.304032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:04.314667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1119/2231] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:14:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:04.331837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:04.344896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52f62c-c5e7-43b8-92fb-8c6fc77f9acc service swift] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:04.354133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb6907-7487-4d40-a6d8-276ea164ed78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:04.355026 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6907-7487-4d40-a6d8-276ea164ed78 service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:04.355476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6907-7487-4d40-a6d8-276ea164ed78 service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:04.355972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6907-7487-4d40-a6d8-276ea164ed78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:04.365950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb6907-7487-4d40-a6d8-276ea164ed78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:04.367984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb6907-7487-4d40-a6d8-276ea164ed78 service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:04.380288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1113/2232] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:05.899871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:05.901963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:06.037444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c0f34-f969-48a8-9628-f2ad933927ab None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:06.064494 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:06.065259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:06.065990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:06.066472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] PATH_INFO: `/v3/users/72d3f543c59340edbc8ad1f3d95161b9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:06.068451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d3f543c59340edbc8ad1f3d95161b9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:06.071259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:06.234382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d3f543c59340edbc8ad1f3d95161b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:06.260710 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d3f543c59340edbc8ad1f3d95161b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:06.300317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d3f543c59340edbc8ad1f3d95161b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:06.301614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d3f543c59340edbc8ad1f3d95161b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:06.325409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d3f543c59340edbc8ad1f3d95161b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:06.342481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1120/2233] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:14:05 2022] DELETE /identity/v3/users/72d3f543c59340edbc8ad1f3d95161b9 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:06.347103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:06.348204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:06.348701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:06.378956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:06.380739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:06.474472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1114/2234] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:06.493464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:06.495022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:06.594372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:06.653926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4a07c-caa4-4063-9260-b2ff2e223d24 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:06.663020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b89f8-8be8-4a2a-b93a-8fe33f3be5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:06.664045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b89f8-8be8-4a2a-b93a-8fe33f3be5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:06.664366 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b89f8-8be8-4a2a-b93a-8fe33f3be5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:06.664614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b89f8-8be8-4a2a-b93a-8fe33f3be5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:06.695951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b89f8-8be8-4a2a-b93a-8fe33f3be5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:06.697746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b89f8-8be8-4a2a-b93a-8fe33f3be5b8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:06.720690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1121/2235] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:08.037191 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:08.039360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:08.064890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe9d10-8e39-4c7b-ba7c-59fcc4c15224 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:08.081144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:08.082569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:08.083263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:08.084688 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] PATH_INFO: `/v3/projects/d1f67b17cc4842c69c4c9d6b73991405` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:08.087034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f67b17cc4842c69c4c9d6b73991405)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:08.094491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:08.192204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:08.215446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f67b17cc4842c69c4c9d6b73991405', 'request_id': 'req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:08.223108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f67b17cc4842c69c4c9d6b73991405', 'request_id': 'req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:08.245994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f67b17cc4842c69c4c9d6b73991405', 'request_id': 'req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:08.270924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] The token cache is being invalidate because project d1f67b17cc4842c69c4c9d6b73991405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:14:08.272079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:08.272926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4b235b91-b7c6-4b3f-8f99-d3428757b1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:08.274901 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1115/2236] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:14:08 2022] DELETE /identity/v3/projects/d1f67b17cc4842c69c4c9d6b73991405 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:08.279595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8797f1-caed-45f2-95ab-0dd97d5f5392 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:08.280131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8797f1-caed-45f2-95ab-0dd97d5f5392 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:08.280493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8797f1-caed-45f2-95ab-0dd97d5f5392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:08.321650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0b8797f1-caed-45f2-95ab-0dd97d5f5392 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:08.322917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b8797f1-caed-45f2-95ab-0dd97d5f5392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:08.476457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1122/2237] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:08.478135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e46bd4-3a40-4141-9179-b1ce5b317d1a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:08.478435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e46bd4-3a40-4141-9179-b1ce5b317d1a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:08.478714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e46bd4-3a40-4141-9179-b1ce5b317d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:08.527067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d9e46bd4-3a40-4141-9179-b1ce5b317d1a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:08.527067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d9e46bd4-3a40-4141-9179-b1ce5b317d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:08.546012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1116/2238] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:08.550055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8797f1-caed-45f2-95ab-0dd97d5f5392 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:08.564375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8797f1-caed-45f2-95ab-0dd97d5f5392 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:08.570154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:08.570780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:08.571045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:08.571321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:08.572047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:14:08.579776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:08.579776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:08.596636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1123/2239] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:14:08.596893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e46bd4-3a40-4141-9179-b1ce5b317d1a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:08.607988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e46bd4-3a40-4141-9179-b1ce5b317d1a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:08.622692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:08.623332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:08.623621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:08.623931 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:08.624579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:08.626503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:08.727141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1117/2240] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:14:08 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:14:08.733190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:08.747004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5124-8a49-4fb9-a4d1-6ac5dd82c52c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:09.132061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:09.132061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:09.132061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:09.132061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:09.132513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:09.132513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:09.186876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1124/2241] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:14:08 2022] POST /identity/v3/users => generated 288 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:14:09.188938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:09.221028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7b5c8-37a0-4346-8d12-df30b6da9f79 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:09.248625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:09.249541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:09.249971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:09.250390 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:09.251302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:09.264454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:09.292034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:09.293151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1118/2242] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:14:09.310185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795686-3929-441f-9787-e3ea8f92e454 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:09.326390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f8112-d4c6-490a-be08-bdb872c1b6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:09.327020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8112-d4c6-490a-be08-bdb872c1b6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:09.327304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8112-d4c6-490a-be08-bdb872c1b6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:09.327583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8112-d4c6-490a-be08-bdb872c1b6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:09.328343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f8112-d4c6-490a-be08-bdb872c1b6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:09.330126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f8112-d4c6-490a-be08-bdb872c1b6b0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:09.361905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1125/2243] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:14:09.363261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:09.379065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c987807-36a6-43ec-a9c9-a453af412a6a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:09.391403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:09.393405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:09.393770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:09.394207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] PATH_INFO: `/v3/projects/cffd22b3b78f416ab10fd92ac92a5589/users/f67c48bdc1644230a97a6d9b8fe8f9f3/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:09.413577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffd22b3b78f416ab10fd92ac92a5589, user_id=f67c48bdc1644230a97a6d9b8fe8f9f3, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:09.416282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:09.441432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1119/2244] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:14:09 2022] PUT /identity/v3/projects/cffd22b3b78f416ab10fd92ac92a5589/users/f67c48bdc1644230a97a6d9b8fe8f9f3/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:14:09.461406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e8da-ff48-4541-927d-ee451cb787a8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:09.461704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e8da-ff48-4541-927d-ee451cb787a8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:09.462003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e8da-ff48-4541-927d-ee451cb787a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:09.563983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c498e8da-ff48-4541-927d-ee451cb787a8 None None] MFA Rules not processed for user `f67c48bdc1644230a97a6d9b8fe8f9f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:09.564918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c498e8da-ff48-4541-927d-ee451cb787a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:09.753130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:09.842958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1126/2245] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:09.851421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:09.853638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:10.008509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036142-abff-4a7c-ae9b-5b438fbedfbf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:10.016637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9ae1d-393c-4c8a-9472-13a0a9a86ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:10.017509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ae1d-393c-4c8a-9472-13a0a9a86ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:10.018138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ae1d-393c-4c8a-9472-13a0a9a86ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:10.018768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ae1d-393c-4c8a-9472-13a0a9a86ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:10.051546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9ae1d-393c-4c8a-9472-13a0a9a86ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:10.053590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9ae1d-393c-4c8a-9472-13a0a9a86ba5 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:10.069326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1120/2246] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:12.007687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:12.009784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:12.010478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:12.083663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda599-049f-4282-8067-752524f33084 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:12.085142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda599-049f-4282-8067-752524f33084 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:12.085744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda599-049f-4282-8067-752524f33084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:12.092918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] MFA Rules not processed for user `f67c48bdc1644230a97a6d9b8fe8f9f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:12.103035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:12.122566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dfcda599-049f-4282-8067-752524f33084 None None] MFA Rules not processed for user `703464e4a9524efaa8fdfc1870158cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:12.130420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dfcda599-049f-4282-8067-752524f33084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:12.130650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1127/2247] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:12.226711 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1121/2248] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:15.800791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:15.802365 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:15.897717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02075c67-6e2e-46b3-9382-45751dfea9fd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:15.903301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:15.904088 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:15.904583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:15.904948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] PATH_INFO: `/v3/users/dc8b7254fd7644418442a0a7619e8188` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:15.906525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8b7254fd7644418442a0a7619e8188)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:15.908842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:15.971532 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8b7254fd7644418442a0a7619e8188'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:15.981562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8b7254fd7644418442a0a7619e8188'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.008388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8b7254fd7644418442a0a7619e8188'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.010033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8b7254fd7644418442a0a7619e8188'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.023840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8b7254fd7644418442a0a7619e8188'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.041770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1128/2249] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:14:15 2022] DELETE /identity/v3/users/dc8b7254fd7644418442a0a7619e8188 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:16.047047 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:16.047610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:16.047610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:16.063622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:16.078529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:16.080223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:16.155291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1122/2250] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:16.166678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:16.169014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:16.239004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:16.247376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45918f-c233-4ffb-a095-94404285d31c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:16.253390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a0b3-ed4f-40d2-9969-dc5d39eb8650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:16.254213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a0b3-ed4f-40d2-9969-dc5d39eb8650 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:16.254517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a0b3-ed4f-40d2-9969-dc5d39eb8650 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:16.254990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a0b3-ed4f-40d2-9969-dc5d39eb8650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:16.267791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a0b3-ed4f-40d2-9969-dc5d39eb8650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:16.269290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a0b3-ed4f-40d2-9969-dc5d39eb8650 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:16.280598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1129/2251] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:16.644023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:16.646449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:16.673928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3df597-c088-47db-bf9c-b2379d9f5fc3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:16.683040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:16.684536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:16.685884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:16.686350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] PATH_INFO: `/v3/projects/3d234f0dfc3547e19a0900b5e916c747` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:16.688460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d234f0dfc3547e19a0900b5e916c747)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:16.691673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:16.778744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d234f0dfc3547e19a0900b5e916c747', 'request_id': 'req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.794199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d234f0dfc3547e19a0900b5e916c747', 'request_id': 'req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.816558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d234f0dfc3547e19a0900b5e916c747', 'request_id': 'req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.874661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] The token cache is being invalidate because project 3d234f0dfc3547e19a0900b5e916c747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:14:16.874959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.875628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fb4aa17b-f1e9-44a3-ae76-b683dda02bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:16.877745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1123/2252] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:14:16 2022] DELETE /identity/v3/projects/3d234f0dfc3547e19a0900b5e916c747 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:16.882462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faf5bf-a7cf-4464-94aa-93eca47048ee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:16.883256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faf5bf-a7cf-4464-94aa-93eca47048ee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:16.883808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faf5bf-a7cf-4464-94aa-93eca47048ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:16.927336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-17faf5bf-a7cf-4464-94aa-93eca47048ee None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:16.929940 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-17faf5bf-a7cf-4464-94aa-93eca47048ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:17.046620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1130/2253] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:17.056975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba83664-c9a1-4510-8400-2152184ee8d7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.059064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba83664-c9a1-4510-8400-2152184ee8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.059788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba83664-c9a1-4510-8400-2152184ee8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.096498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4ba83664-c9a1-4510-8400-2152184ee8d7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:17.098005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ba83664-c9a1-4510-8400-2152184ee8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:17.115889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1124/2254] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:17.120036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faf5bf-a7cf-4464-94aa-93eca47048ee None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:17.132960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faf5bf-a7cf-4464-94aa-93eca47048ee None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:17.148263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:17.148896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.149183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.149444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.153636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:14:17.153957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:17.155575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:17.185515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1131/2255] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:14:17.189997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba83664-c9a1-4510-8400-2152184ee8d7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:17.202444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba83664-c9a1-4510-8400-2152184ee8d7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:17.207452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:17.208768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.209142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.209511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.211276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:17.213339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:17.273093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1125/2256] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:14:17 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:14:17.277154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:17.290729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632de6f-244a-4eb6-b122-ec1149f4eab4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:17.405074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:17.405798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.406130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.406414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.407101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:17.409379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:17.435034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1132/2257] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:14:17 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:14:17.442216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:17.459281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e0255-5f46-43cf-8860-da8d77a55683 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:17.466244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:17.466937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.467277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.467611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.468519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:17.471568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:17.486160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1126/2258] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:14:17.487676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:17.499076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088717af-0bb4-4b5e-8e10-7909f6e04999 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:17.504341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461e64b-a1ae-43a0-882b-40626b156f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:17.505913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461e64b-a1ae-43a0-882b-40626b156f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.506304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461e64b-a1ae-43a0-882b-40626b156f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.506673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461e64b-a1ae-43a0-882b-40626b156f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.508050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461e64b-a1ae-43a0-882b-40626b156f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:17.509187 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461e64b-a1ae-43a0-882b-40626b156f38 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:17.528006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1133/2259] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:14:17.532211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:17.544974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03d141-6f1f-43da-b74a-5ae35d2096dc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:17.555016 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:17.555869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.556303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.556627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] PATH_INFO: `/v3/projects/25d31e6f0604458f8b9a36e2de86ff1f/users/18c5c95fde3646409d542da0ff274883/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.585687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d31e6f0604458f8b9a36e2de86ff1f, user_id=18c5c95fde3646409d542da0ff274883, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:17.588583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:17.603501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1127/2260] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:14:17 2022] PUT /identity/v3/projects/25d31e6f0604458f8b9a36e2de86ff1f/users/18c5c95fde3646409d542da0ff274883/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:17.608693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863775ee-40a5-49c0-bafb-747aa1e3f943 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.609304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863775ee-40a5-49c0-bafb-747aa1e3f943 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.609800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863775ee-40a5-49c0-bafb-747aa1e3f943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.633987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-863775ee-40a5-49c0-bafb-747aa1e3f943 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:17.635707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-863775ee-40a5-49c0-bafb-747aa1e3f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:17.697570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:17.715260 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1134/2261] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:14:17.727728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:17.729690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:17.771217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:17.816285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c83ba2-f9b5-4660-ac8c-ebccfb93a7c3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:17.826932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf57f-1c68-4184-85f0-c327f76f8e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:17.827797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf57f-1c68-4184-85f0-c327f76f8e4b service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:17.827985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf57f-1c68-4184-85f0-c327f76f8e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:17.828449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf57f-1c68-4184-85f0-c327f76f8e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:17.841462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bf57f-1c68-4184-85f0-c327f76f8e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:17.842985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bf57f-1c68-4184-85f0-c327f76f8e4b service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:17.854548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1128/2262] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:22.957149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9af41-144c-48be-b8f9-e9561cd3af22 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:22.957784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9af41-144c-48be-b8f9-e9561cd3af22 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:22.957784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9af41-144c-48be-b8f9-e9561cd3af22 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:22.959206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1135/2263] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:14:22.969292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d9576-ab1c-484f-a334-5eee1d79bc69 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:22.970276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d9576-ab1c-484f-a334-5eee1d79bc69 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:22.970816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d9576-ab1c-484f-a334-5eee1d79bc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:22.997694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-198d9576-ab1c-484f-a334-5eee1d79bc69 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:22.999399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-198d9576-ab1c-484f-a334-5eee1d79bc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:23.175024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9af41-144c-48be-b8f9-e9561cd3af22 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:23.176842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1129/2264] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:14:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:23.180092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99d9af41-144c-48be-b8f9-e9561cd3af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:23.263979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9af41-144c-48be-b8f9-e9561cd3af22 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:23.273151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163093-9e53-40a0-b04e-717004b0cfd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:23.274058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163093-9e53-40a0-b04e-717004b0cfd5 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:23.274498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163093-9e53-40a0-b04e-717004b0cfd5 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:23.274897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163093-9e53-40a0-b04e-717004b0cfd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:23.294195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10163093-9e53-40a0-b04e-717004b0cfd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:23.297864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10163093-9e53-40a0-b04e-717004b0cfd5 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:23.321295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1136/2265] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:23.441528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9267e-0144-4964-8ef4-0be726387fea None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:23.441528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9267e-0144-4964-8ef4-0be726387fea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:23.441911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9267e-0144-4964-8ef4-0be726387fea None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:23.445121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1130/2266] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:14:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:14:23.458487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d39833-ba4f-466a-91a7-6d7ffe9d1451 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:23.458820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d39833-ba4f-466a-91a7-6d7ffe9d1451 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:23.459167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d39833-ba4f-466a-91a7-6d7ffe9d1451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:23.508897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-34d39833-ba4f-466a-91a7-6d7ffe9d1451 None None] MFA Rules not processed for user `bc47f8d295ed481a87b4fb0d405c605e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:23.523838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-34d39833-ba4f-466a-91a7-6d7ffe9d1451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:23.548405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1137/2267] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:23.597214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9267e-0144-4964-8ef4-0be726387fea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:23.615125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9267e-0144-4964-8ef4-0be726387fea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:23.624653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47586cab-31e3-49b6-b9d5-322de72a8bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:23.625366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47586cab-31e3-49b6-b9d5-322de72a8bed service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:23.625778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47586cab-31e3-49b6-b9d5-322de72a8bed service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:23.626172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47586cab-31e3-49b6-b9d5-322de72a8bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:23.652529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47586cab-31e3-49b6-b9d5-322de72a8bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:23.654205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47586cab-31e3-49b6-b9d5-322de72a8bed service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:23.676103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1131/2268] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:24.110343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77289e-6563-4b5e-869f-0a3eb826074a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:24.111374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77289e-6563-4b5e-869f-0a3eb826074a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:24.111995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77289e-6563-4b5e-869f-0a3eb826074a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:24.113606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1138/2269] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:14:24.136707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279c14-8627-475e-b1e1-d23cdbfe0907 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:24.138272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279c14-8627-475e-b1e1-d23cdbfe0907 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:24.139248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279c14-8627-475e-b1e1-d23cdbfe0907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:24.181327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b8279c14-8627-475e-b1e1-d23cdbfe0907 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:24.183012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b8279c14-8627-475e-b1e1-d23cdbfe0907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:24.195348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1132/2270] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:14:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:24.212956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77289e-6563-4b5e-869f-0a3eb826074a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:24.235359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77289e-6563-4b5e-869f-0a3eb826074a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:24.248305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:24.249838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:24.251487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:24.252135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:24.278145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:24.280999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:24.310487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1139/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:31.228121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a395-f9de-4344-a80d-e8f1ee11cadd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:31.230567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a395-f9de-4344-a80d-e8f1ee11cadd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:31.231098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a395-f9de-4344-a80d-e8f1ee11cadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:31.283479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2e01a395-f9de-4344-a80d-e8f1ee11cadd None None] MFA Rules not processed for user `18c5c95fde3646409d542da0ff274883`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:31.284957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2e01a395-f9de-4344-a80d-e8f1ee11cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:31.378355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1133/2272] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:31.383240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:31.396086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c7485-ded4-4874-989d-6f77ebd144e8 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:31.401947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:31.402929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:31.403429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:31.403959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:31.404827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:31.407267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:31.468054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1140/2273] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:14:31 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:14:31.472331 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01a395-f9de-4344-a80d-e8f1ee11cadd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:31.485023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01a395-f9de-4344-a80d-e8f1ee11cadd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:31.534929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:31.568503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:31.569376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:31.569666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:31.569924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:31.570553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:31.572728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:31.601633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1134/2274] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:14:31 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:14:31.602792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:31.613264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637a575-c4f2-4044-a7f4-79c01eec9bf3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:31.618508 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:31.619148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:31.619435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:31.619891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:31.620526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:31.622149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:31.633074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1141/2275] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:14:31.637529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:31.649015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d745904-72c2-4bcc-9246-07cc7bb64b79 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:31.655914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:31.656601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:31.656958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:31.657137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] PATH_INFO: `/v3/projects/5b79817953084f4da300b677a7ed4f3f/users/46119bc0364843aaa93153b13d22e48d/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:31.689306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b79817953084f4da300b677a7ed4f3f, user_id=46119bc0364843aaa93153b13d22e48d, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:31.694591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:31.716813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1135/2276] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:14:31 2022] PUT /identity/v3/projects/5b79817953084f4da300b677a7ed4f3f/users/46119bc0364843aaa93153b13d22e48d/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:14:31.720616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:31.731472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba1b15-57bc-46e0-8d07-6aefd8351082 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:31.800801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7faea-1984-4232-9a39-ae4f630dd155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:31.801455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7faea-1984-4232-9a39-ae4f630dd155 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:31.801734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7faea-1984-4232-9a39-ae4f630dd155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:31.802009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7faea-1984-4232-9a39-ae4f630dd155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:31.802894 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7faea-1984-4232-9a39-ae4f630dd155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:31.804247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7faea-1984-4232-9a39-ae4f630dd155 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:31.814268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1142/2277] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:14:31.818500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:31.829082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d9f75-1103-4e97-9bf5-1cbda72252ea admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:31.834550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd229e-4d29-489c-bad9-2a3dccf92b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:31.835305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd229e-4d29-489c-bad9-2a3dccf92b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:31.835655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd229e-4d29-489c-bad9-2a3dccf92b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:31.836092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd229e-4d29-489c-bad9-2a3dccf92b23 admin admin] PATH_INFO: `/v3/projects/5b79817953084f4da300b677a7ed4f3f/users/46119bc0364843aaa93153b13d22e48d/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:31.838759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd229e-4d29-489c-bad9-2a3dccf92b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b79817953084f4da300b677a7ed4f3f, user_id=46119bc0364843aaa93153b13d22e48d, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:31.841429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd229e-4d29-489c-bad9-2a3dccf92b23 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:31.852498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1136/2278] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:14:31 2022] PUT /identity/v3/projects/5b79817953084f4da300b677a7ed4f3f/users/46119bc0364843aaa93153b13d22e48d/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:34.895702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:34.896195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:34.899213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:34.941927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] MFA Rules not processed for user `2d2b7b4cb4a34964a4d6127a1ed3ac98`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:34.943861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:35.049686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:35.077028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1143/2279] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:35.147081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5df1-9829-4bfe-ae2f-12ae9e53a43d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:35.147600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5df1-9829-4bfe-ae2f-12ae9e53a43d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:35.147696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5df1-9829-4bfe-ae2f-12ae9e53a43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:35.180821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-671c5df1-9829-4bfe-ae2f-12ae9e53a43d None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:35.182250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-671c5df1-9829-4bfe-ae2f-12ae9e53a43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:35.264603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1137/2280] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 6 01:14:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:35.287217 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:35.289760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:35.489328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e5fd8-93f3-4c7b-8ce7-ecd9ca595e94 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:35.504487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:35.506423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:35.507635 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:35.509323 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:35.540075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:35.541858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:35.579081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1144/2281] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:37.043342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:37.043894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:37.044043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:37.083056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:37.084897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:37.134559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:37.136845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1138/2282] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:37.156812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18cc78-db1d-4ca0-b34b-997f0138d4d3 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:37.164673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:37.165392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:37.168925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:37.169216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:37.195632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:37.197400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:37.220206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1145/2283] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:38.189993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:38.192207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:38.384322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776b749-8d76-4cb6-946f-ac7dc62843ac None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:38.397499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:38.398435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:38.398946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:38.399638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:38.402660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:38.420678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:38.424496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:38.455012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1139/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:38.852415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:38.854382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:39.032866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b958a-fa17-49bc-9a0d-6f445c1a1cc8 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:39.050491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:39.051400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:39.051704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:39.052112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] PATH_INFO: `/v3/users/453fa4c58743419bb29e5f01d9f78041` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:39.054742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=453fa4c58743419bb29e5f01d9f78041)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:39.056970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:39.153444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '453fa4c58743419bb29e5f01d9f78041'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:39.169354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '453fa4c58743419bb29e5f01d9f78041'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:39.186245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:39.197142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '453fa4c58743419bb29e5f01d9f78041'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:39.198576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '453fa4c58743419bb29e5f01d9f78041'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:39.230684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8fdc34a1-3f38-499b-bbb9-b12581a14c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '453fa4c58743419bb29e5f01d9f78041'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:39.253070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1146/2285] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:14:38 2022] DELETE /identity/v3/users/453fa4c58743419bb29e5f01d9f78041 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:40.235526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:40.253836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5195e5-9a58-4d0a-ae4f-21a043df8b25 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:40.264214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:40.373039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:40.374086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:40.374581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:40.375041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] PATH_INFO: `/v3/projects/9ead4033c51d4658a5406c045da1c26e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:40.376429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ead4033c51d4658a5406c045da1c26e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:40.378722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:40.492250 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ead4033c51d4658a5406c045da1c26e', 'request_id': 'req-7f998011-a3e6-4a77-ac25-f4f478ea5494'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:40.513462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ead4033c51d4658a5406c045da1c26e', 'request_id': 'req-7f998011-a3e6-4a77-ac25-f4f478ea5494'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:40.545112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ead4033c51d4658a5406c045da1c26e', 'request_id': 'req-7f998011-a3e6-4a77-ac25-f4f478ea5494'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:40.563728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] The token cache is being invalidate because project 9ead4033c51d4658a5406c045da1c26e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:14:40.563728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:40.563728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:40.579014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1140/2286] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:14:40 2022] DELETE /identity/v3/projects/9ead4033c51d4658a5406c045da1c26e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:41.119035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eff69-8ab0-4e71-ab8f-099a084e4bd4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:41.120618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eff69-8ab0-4e71-ab8f-099a084e4bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:41.121179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91eff69-8ab0-4e71-ab8f-099a084e4bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:41.179417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e91eff69-8ab0-4e71-ab8f-099a084e4bd4 None None] MFA Rules not processed for user `46119bc0364843aaa93153b13d22e48d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:41.181199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e91eff69-8ab0-4e71-ab8f-099a084e4bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:41.324239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1147/2287] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:41.350891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:41.352730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:41.447103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998011-a3e6-4a77-ac25-f4f478ea5494 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:41.452599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:41.453234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:41.453543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:41.453786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:41.455650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:41.585272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:41.591517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:41.595293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:41.612292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1141/2288] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:14:41.873657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b63e6-d8dc-4de9-b4f6-48a02dd5b77c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:41.874022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b63e6-d8dc-4de9-b4f6-48a02dd5b77c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:41.874347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b63e6-d8dc-4de9-b4f6-48a02dd5b77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:41.925486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-055b63e6-d8dc-4de9-b4f6-48a02dd5b77c None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:41.927274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-055b63e6-d8dc-4de9-b4f6-48a02dd5b77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:42.063527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1148/2289] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:42.081365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:42.084175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:42.224222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144f60-1804-4224-a136-2e10ca0e6c0a service glance] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:42.231630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b358d-723b-49e1-981a-c78b6b965a3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:42.232457 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b358d-723b-49e1-981a-c78b6b965a3a service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:42.232906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b358d-723b-49e1-981a-c78b6b965a3a service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:42.233333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b358d-723b-49e1-981a-c78b6b965a3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:42.249455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b358d-723b-49e1-981a-c78b6b965a3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:42.251264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b358d-723b-49e1-981a-c78b6b965a3a service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:42.264013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1142/2290] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 01:14:52.048564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=66,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:52.258768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=34 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:53.958799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6f79a-de15-4793-baa4-92ecbe5b62bf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:53.959313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6f79a-de15-4793-baa4-92ecbe5b62bf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:53.959503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6f79a-de15-4793-baa4-92ecbe5b62bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:53.979063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-46e6f79a-de15-4793-baa4-92ecbe5b62bf None None] MFA Rules not processed for user `18c5c95fde3646409d542da0ff274883`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:53.980230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-46e6f79a-de15-4793-baa4-92ecbe5b62bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:53.988156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1149/2291] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:55.168734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2a7d7-f21a-4fd6-adfe-32941ac71d1d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:55.169334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2a7d7-f21a-4fd6-adfe-32941ac71d1d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:55.169334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2a7d7-f21a-4fd6-adfe-32941ac71d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:55.193558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a6d2a7d7-f21a-4fd6-adfe-32941ac71d1d None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:55.194812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a6d2a7d7-f21a-4fd6-adfe-32941ac71d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:55.203072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1143/2292] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:14:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:14:55.219110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6f79a-de15-4793-baa4-92ecbe5b62bf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:55.234400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6f79a-de15-4793-baa4-92ecbe5b62bf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:55.241479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:55.242196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:55.242548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:55.242897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:55.252306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:55.254585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:55.263452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1150/2293] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:55.837022 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a3491-8b8f-4673-a709-33b07c712a03 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:55.837311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a3491-8b8f-4673-a709-33b07c712a03 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:55.837587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a3491-8b8f-4673-a709-33b07c712a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:55.858147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4e1a3491-8b8f-4673-a709-33b07c712a03 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:55.859330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4e1a3491-8b8f-4673-a709-33b07c712a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:55.866720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1144/2294] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 6 01:14:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:55.879519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:55.890077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9880a-2c83-4859-8997-600aca011df5 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:55.895783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:55.896465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:55.896787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:55.897060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:55.904996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:55.906551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:55.915409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1151/2295] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:56.166837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179559-dea6-444c-af93-a43500283458 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:56.167126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179559-dea6-444c-af93-a43500283458 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:56.167403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179559-dea6-444c-af93-a43500283458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:56.186743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7d179559-dea6-444c-af93-a43500283458 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:56.187979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7d179559-dea6-444c-af93-a43500283458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:56.195832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1145/2296] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:14:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:56.208517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:56.218866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdda902-a0ec-4045-a445-9708e64fa741 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:56.223936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:56.224533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:56.224813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:56.225087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:56.233010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:56.234565 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:56.242396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1152/2297] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:56.614158 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acb180-498c-42cc-ba13-b7585b27f73f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:56.614273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acb180-498c-42cc-ba13-b7585b27f73f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:56.614463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acb180-498c-42cc-ba13-b7585b27f73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:56.634601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-32acb180-498c-42cc-ba13-b7585b27f73f None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:56.636132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32acb180-498c-42cc-ba13-b7585b27f73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:56.648746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1146/2298] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 6 01:14:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:56.657099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:56.679836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e908e9a-72b6-4b21-a680-b976757ebfab service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:56.687432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:56.688129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:56.688416 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:56.688721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:56.700051 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:56.708068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:56.728537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1153/2299] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:58.682794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:58.683268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:58.683359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:58.702836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:58.704095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:58.716041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1147/2300] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:14:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:14:58.723840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:58.734501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55b394-dae6-4751-9e8a-5e479508f05a service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:58.741110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb58d0-8c32-49ff-bfc7-3f3ab65ee44f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:58.741912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58d0-8c32-49ff-bfc7-3f3ab65ee44f service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:58.742195 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58d0-8c32-49ff-bfc7-3f3ab65ee44f service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:58.742471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58d0-8c32-49ff-bfc7-3f3ab65ee44f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:58.751406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb58d0-8c32-49ff-bfc7-3f3ab65ee44f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:58.753078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb58d0-8c32-49ff-bfc7-3f3ab65ee44f service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:58.761907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1154/2301] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:14:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:14:59.221074 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:59.222872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:59.373734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c2372-62a9-4e6c-b871-307bdf795b2d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:59.379217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:59.380203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:59.380629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:59.381060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] PATH_INFO: `/v3/users/f67c48bdc1644230a97a6d9b8fe8f9f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:59.382381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67c48bdc1644230a97a6d9b8fe8f9f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:59.384535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:59.473093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67c48bdc1644230a97a6d9b8fe8f9f3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:59.473853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67c48bdc1644230a97a6d9b8fe8f9f3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:59.484598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67c48bdc1644230a97a6d9b8fe8f9f3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:59.507434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67c48bdc1644230a97a6d9b8fe8f9f3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:59.508443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67c48bdc1644230a97a6d9b8fe8f9f3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:14:59.538482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1148/2302] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:14:59 2022] DELETE /identity/v3/users/f67c48bdc1644230a97a6d9b8fe8f9f3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:14:59.540065 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:59.540355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:59.540630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:59.601272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:14:59.602786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:59.735682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1155/2303] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:14:59.765280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:14:59.766744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:14:59.869249 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:14:59.874627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de15466-eef2-4cfb-b7ec-4340eb0836cb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:14:59.881482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc2fa4-032b-4e48-b14e-68c45edd8d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:14:59.882384 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc2fa4-032b-4e48-b14e-68c45edd8d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:14:59.883789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc2fa4-032b-4e48-b14e-68c45edd8d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:14:59.884324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc2fa4-032b-4e48-b14e-68c45edd8d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:14:59.900216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc2fa4-032b-4e48-b14e-68c45edd8d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:14:59.901976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc2fa4-032b-4e48-b14e-68c45edd8d81 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:14:59.921277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1149/2304] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:15:00.223287 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:15:00.224803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:15:00.249385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63f270-3e2e-4c4c-aaf1-a970da5fa24e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:15:00.266406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:15:00.267362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:15:00.267880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:15:00.268385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] PATH_INFO: `/v3/projects/cffd22b3b78f416ab10fd92ac92a5589` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:15:00.270317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=cffd22b3b78f416ab10fd92ac92a5589)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:15:00.272601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:15:00.358935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cffd22b3b78f416ab10fd92ac92a5589', 'request_id': 'req-3aabc45d-1096-4ece-8dd7-977a203b7fde'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:15:00.379965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cffd22b3b78f416ab10fd92ac92a5589', 'request_id': 'req-3aabc45d-1096-4ece-8dd7-977a203b7fde'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:15:00.398808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cffd22b3b78f416ab10fd92ac92a5589', 'request_id': 'req-3aabc45d-1096-4ece-8dd7-977a203b7fde'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:15:00.422656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] The token cache is being invalidate because project cffd22b3b78f416ab10fd92ac92a5589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:15:00.423008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:15:00.423676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:15:00.426639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1156/2305] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:15:00 2022] DELETE /identity/v3/projects/cffd22b3b78f416ab10fd92ac92a5589 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:15:09.911268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:15:10.412870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:15:11.344919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbe9b-a516-4030-8f60-323f7b2644f4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:15:11.345380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbe9b-a516-4030-8f60-323f7b2644f4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:15:11.345482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbe9b-a516-4030-8f60-323f7b2644f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:15:11.365462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cecfbe9b-a516-4030-8f60-323f7b2644f4 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:15:11.366594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cecfbe9b-a516-4030-8f60-323f7b2644f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:15:11.420708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1150/2306] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 6 01:15:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:15:11.468174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:15:11.469728 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:15:11.547290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc45d-1096-4ece-8dd7-977a203b7fde admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:15:11.553349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:15:11.553966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:15:11.554244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:15:11.554523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:15:11.563139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:15:11.564837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:15:11.574213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1157/2307] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:15:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:15:21.413140 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=25 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:15:21.570859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:15:34.777796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:15:34.778295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:15:34.778401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:15:34.798433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] MFA Rules not processed for user `f8f44df824414e5ebc8cfd2126e9d66f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:15:34.799689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:15:34.861423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1151/2308] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:15:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:15:37.311717 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:15:37.313201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:15:37.370145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3f6c1-0704-40b2-bd88-f92de002837f service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:15:37.373658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:15:37.374256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:15:37.374533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:15:37.374807 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:15:37.376534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:15:37.437286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:15:37.438702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:15:37.447937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1158/2309] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:15:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:15:44.852245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=25 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:15:47.443804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=50 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:07.424083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:07.426305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:07.514304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdad1e0-af88-497f-8455-922a4b12512f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:07.518820 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:07.520301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:07.520540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:07.520771 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] PATH_INFO: `/v3/users/2d2b7b4cb4a34964a4d6127a1ed3ac98` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:07.521917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2b7b4cb4a34964a4d6127a1ed3ac98)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:07.523867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:07.585293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2b7b4cb4a34964a4d6127a1ed3ac98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:07.586096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2b7b4cb4a34964a4d6127a1ed3ac98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:07.596389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2b7b4cb4a34964a4d6127a1ed3ac98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:07.614551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2b7b4cb4a34964a4d6127a1ed3ac98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:07.615337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d05f6b0f-f8f8-4942-b99c-bf293021bd53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2b7b4cb4a34964a4d6127a1ed3ac98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:07.629233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1152/2310] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:16:07 2022] DELETE /identity/v3/users/2d2b7b4cb4a34964a4d6127a1ed3ac98 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:16:08.014337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.026902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e87ba-ae79-4113-afd6-6e38a5f7080a service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.098990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.099737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.100044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.100391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] PATH_INFO: `/v3/projects/d4d649e0b0434716b8d4aaf4ea171a8e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.101609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d649e0b0434716b8d4aaf4ea171a8e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.103775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.161374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d649e0b0434716b8d4aaf4ea171a8e', 'request_id': 'req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:08.167484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d649e0b0434716b8d4aaf4ea171a8e', 'request_id': 'req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:08.175609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d649e0b0434716b8d4aaf4ea171a8e', 'request_id': 'req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:08.187111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] The token cache is being invalidate because project d4d649e0b0434716b8d4aaf4ea171a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:16:08.187400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:08.188054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e6453540-3a62-4d3e-b9c0-8fbdb73390fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:08.189526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1159/2311] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:16:08 2022] DELETE /identity/v3/projects/d4d649e0b0434716b8d4aaf4ea171a8e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:16:08.194572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e7a7e-62a9-4872-81a9-29a2db2d5b7c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.194817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e7a7e-62a9-4872-81a9-29a2db2d5b7c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.195088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e7a7e-62a9-4872-81a9-29a2db2d5b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.214641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c44e7a7e-62a9-4872-81a9-29a2db2d5b7c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:16:08.215883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c44e7a7e-62a9-4872-81a9-29a2db2d5b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:08.278511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1153/2312] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:16:08.286951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978c816-4bfc-4d94-8c15-1d48eb4fda7f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.287240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978c816-4bfc-4d94-8c15-1d48eb4fda7f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.287516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978c816-4bfc-4d94-8c15-1d48eb4fda7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.308036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d978c816-4bfc-4d94-8c15-1d48eb4fda7f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:16:08.309235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d978c816-4bfc-4d94-8c15-1d48eb4fda7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:08.317032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1160/2313] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:16:08.321909 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e7a7e-62a9-4872-81a9-29a2db2d5b7c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.331176 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e7a7e-62a9-4872-81a9-29a2db2d5b7c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.336002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.336610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.337381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.337981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.338646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:16:08.338947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.340605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.354374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1154/2314] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:16:08.358870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978c816-4bfc-4d94-8c15-1d48eb4fda7f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.369188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978c816-4bfc-4d94-8c15-1d48eb4fda7f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.374024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.374690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.375038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.375377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.376133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.378097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.430207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1161/2315] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:16:08 2022] POST /identity/v3/projects => generated 399 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:16:08.432082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.446460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5375d-35fb-4a3f-9d7e-0d2fc9a90eba admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.451876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:08.537048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.537679 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.537949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.538219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.538863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.540885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.568371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1155/2316] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:16:08 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:16:08.572687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.586061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c646fe2-d9ed-4eb7-8323-f5fa0939a505 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.592555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.593185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.593516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.593788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.594435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.596254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.609368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1162/2317] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:16:08.613804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.624077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f523a-0f69-467b-95c7-8b276f7d9fa6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.629499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bb1f3-53a7-4af3-b319-5d04fb2eb798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.630111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bb1f3-53a7-4af3-b319-5d04fb2eb798 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.630387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bb1f3-53a7-4af3-b319-5d04fb2eb798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.630654 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bb1f3-53a7-4af3-b319-5d04fb2eb798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.631291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bb1f3-53a7-4af3-b319-5d04fb2eb798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.632938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bb1f3-53a7-4af3-b319-5d04fb2eb798 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.644611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1156/2318] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:16:08.648855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.660867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4776689-a248-4768-be78-9665f72ffe9e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.667245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.668131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.668515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.668896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] PATH_INFO: `/v3/projects/b7729e293518490c966263095e617bc8/users/a13abcfebbcc4931be2144e99225c451/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.688379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7729e293518490c966263095e617bc8, user_id=a13abcfebbcc4931be2144e99225c451, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.691661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.708528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1163/2319] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:16:08 2022] PUT /identity/v3/projects/b7729e293518490c966263095e617bc8/users/a13abcfebbcc4931be2144e99225c451/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:16:08.718469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16166-4eaa-4075-897b-e084ca8006e5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.718682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16166-4eaa-4075-897b-e084ca8006e5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.718950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16166-4eaa-4075-897b-e084ca8006e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.752278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e6a16166-4eaa-4075-897b-e084ca8006e5 None None] MFA Rules not processed for user `a13abcfebbcc4931be2144e99225c451`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:16:08.754232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e6a16166-4eaa-4075-897b-e084ca8006e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:08.832509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1157/2320] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:16:08.844412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:08.848172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:08.892423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:08.922258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe193f-f694-4b16-9fa3-c7ca91bc7f5b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:08.926495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:08.927100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:08.927381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:08.927654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:08.936172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:08.937594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:08.946364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1164/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:16:12.042864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3adcce-c3ca-4adc-8e55-24c2fd54e579 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:12.043424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3adcce-c3ca-4adc-8e55-24c2fd54e579 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:12.043560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3adcce-c3ca-4adc-8e55-24c2fd54e579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:12.066192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3c3adcce-c3ca-4adc-8e55-24c2fd54e579 None None] MFA Rules not processed for user `2dc5c46544a447159b7fd3d225cd9697`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:16:12.067339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3c3adcce-c3ca-4adc-8e55-24c2fd54e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:12.130515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1158/2322] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:16:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:16:16.144870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:16.232501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e13b61-ff2f-4bf0-8be7-4fd56afe7d55 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:16.238636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:16.238636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:16.238935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:16.239206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] PATH_INFO: `/v3/users/a13abcfebbcc4931be2144e99225c451` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:16.240349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13abcfebbcc4931be2144e99225c451)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:16.242305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:16.290816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13abcfebbcc4931be2144e99225c451'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:16.298523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13abcfebbcc4931be2144e99225c451'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:16.311133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13abcfebbcc4931be2144e99225c451'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:16.311776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13abcfebbcc4931be2144e99225c451'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:16.318556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13abcfebbcc4931be2144e99225c451'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:16.324315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1165/2323] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:16:16 2022] DELETE /identity/v3/users/a13abcfebbcc4931be2144e99225c451 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:16:16.328726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:16.329196 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:16.329292 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:16.348248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:16:16.349435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:16.361713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:16.408373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1159/2324] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:16:16.418824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:16.420276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:16.471726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:16.476371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935a318-d092-467f-af8b-e1f002bed424 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:16.480997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06fe71-ecd9-4e39-82dd-41eb58e36ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:16.481611 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06fe71-ecd9-4e39-82dd-41eb58e36ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:16.482111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06fe71-ecd9-4e39-82dd-41eb58e36ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:16.482549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06fe71-ecd9-4e39-82dd-41eb58e36ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:16.492699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06fe71-ecd9-4e39-82dd-41eb58e36ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:16.494109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06fe71-ecd9-4e39-82dd-41eb58e36ce0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:16.503564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1166/2325] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:16:16.918704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:16.921116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:16.933496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdaa0a-a5ac-4a4b-9d00-f8e023d791e3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:16.962772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06941882-1600-4618-ae0f-076892986750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:16.963571 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06941882-1600-4618-ae0f-076892986750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:16.964462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06941882-1600-4618-ae0f-076892986750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:16.965210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06941882-1600-4618-ae0f-076892986750 admin admin] PATH_INFO: `/v3/projects/b7729e293518490c966263095e617bc8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:16.967149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06941882-1600-4618-ae0f-076892986750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7729e293518490c966263095e617bc8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:16.969452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06941882-1600-4618-ae0f-076892986750 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:17.023342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7729e293518490c966263095e617bc8', 'request_id': 'req-06941882-1600-4618-ae0f-076892986750'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:17.036227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7729e293518490c966263095e617bc8', 'request_id': 'req-06941882-1600-4618-ae0f-076892986750'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:17.048842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7729e293518490c966263095e617bc8', 'request_id': 'req-06941882-1600-4618-ae0f-076892986750'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:17.060977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-06941882-1600-4618-ae0f-076892986750 admin admin] The token cache is being invalidate because project b7729e293518490c966263095e617bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:16:17.061295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:17.061941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:16:17.063629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1160/2326] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:16:16 2022] DELETE /identity/v3/projects/b7729e293518490c966263095e617bc8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:16:18.555312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23e1e3-e159-4af5-a41f-c88e0e2aa70b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:18.555874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23e1e3-e159-4af5-a41f-c88e0e2aa70b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:18.555874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23e1e3-e159-4af5-a41f-c88e0e2aa70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:18.575606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2c23e1e3-e159-4af5-a41f-c88e0e2aa70b None None] MFA Rules not processed for user `703464e4a9524efaa8fdfc1870158cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:16:18.576856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c23e1e3-e159-4af5-a41f-c88e0e2aa70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:18.640412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1167/2327] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:16:26.690195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:26.693353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:26.768797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06941882-1600-4618-ae0f-076892986750 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:26.773199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:26.773816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:26.774101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:26.774373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:26.776336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:26.837500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:26.861205 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:26.863005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:26.875113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1161/2328] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:16:27.362184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23e1e3-e159-4af5-a41f-c88e0e2aa70b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:27.372680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23e1e3-e159-4af5-a41f-c88e0e2aa70b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:27.376397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8838c-8887-40f8-85b2-a67205f46007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:27.377013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8838c-8887-40f8-85b2-a67205f46007 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:27.377295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8838c-8887-40f8-85b2-a67205f46007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:27.377619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8838c-8887-40f8-85b2-a67205f46007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:27.380671 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ea8838c-8887-40f8-85b2-a67205f46007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:27.395336 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8838c-8887-40f8-85b2-a67205f46007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:27.396485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8838c-8887-40f8-85b2-a67205f46007 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:27.408437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1168/2329] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:16:36.869364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:37.058358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:37.059746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:37.073692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a06d0b-5df1-44a4-ae8b-e55a6cfabfc6 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:37.078971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:37.079561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:37.079875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:37.080145 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:37.081751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:37.139674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:37.141236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:37.150209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1162/2330] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:16:37.402847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=35 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:47.146161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=28 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:16:54.430621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:54.431127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:54.431224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:54.450155 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] MFA Rules not processed for user `f8f44df824414e5ebc8cfd2126e9d66f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:16:54.451357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:54.458192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1169/2331] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:16:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:16:54.757747 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:54.768699 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce97ba-aed0-4ca2-bc0e-85b7258e1a93 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:54.773715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:54.774309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:54.774584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:54.774852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:54.776610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:54.786040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:54.787436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:54.795290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1163/2332] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:16:55.100873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:55.102468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:55.174654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1604c1e-249f-4f45-8292-aacd46bf7e76 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:55.180024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:55.180642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:55.180901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:55.181169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:55.182843 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:55.195196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:55.196863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:55.206702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1170/2333] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:16:57.143136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:16:57.144781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:57.155452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8b046-aa1d-4585-9af5-f442f9be2a56 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:16:57.159702 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:16:57.160339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:16:57.160615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:16:57.160883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:16:57.164639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:16:57.175041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:16:57.176530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:16:57.186745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1164/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:16:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:02.965951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:02.967354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:03.022142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38f9d6-73f2-4f46-9283-e31392c3ba46 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:03.025719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:03.026311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:03.026595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:03.026868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:03.028546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:03.039874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:03.041280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:03.050263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1171/2335] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:07.182071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=12 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:10.281638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:10.283164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:10.300750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc385-1772-4109-b24b-3ec28c059488 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:10.307573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:10.308411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:10.308872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:10.309307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:10.311310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:10.328240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:10.330028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:10.342015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1165/2336] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:10.487040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:10.508339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd72ac-3d04-4249-8465-1dd870981eaa service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:10.526993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c93db-a535-411f-84bc-e91bd4917c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:10.528554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c93db-a535-411f-84bc-e91bd4917c5e service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:10.529036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c93db-a535-411f-84bc-e91bd4917c5e service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:10.529495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c93db-a535-411f-84bc-e91bd4917c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:10.542053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a8c93db-a535-411f-84bc-e91bd4917c5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:10.560593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c93db-a535-411f-84bc-e91bd4917c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:10.562408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c93db-a535-411f-84bc-e91bd4917c5e service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:10.583558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1172/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:11.023000 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:11.035009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c685a-cb69-4ae4-840c-326a75ad3d3f service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:11.042845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:11.044192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:11.044660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:11.045087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:11.047105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:11.122455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:11.124092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:11.135143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1166/2338] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:20.569999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=67 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:21.129764 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=34 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:21.702679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100ca7c-08fe-4557-857c-a5459fdde7b7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:21.703150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100ca7c-08fe-4557-857c-a5459fdde7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:21.703298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100ca7c-08fe-4557-857c-a5459fdde7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:21.722078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c100ca7c-08fe-4557-857c-a5459fdde7b7 None None] MFA Rules not processed for user `e17a4e664b2c4ac9a9fea65287faad95`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:21.723277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c100ca7c-08fe-4557-857c-a5459fdde7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:21.778871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1173/2339] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:22.084992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:22.094009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0311-3ad2-48eb-a1ec-613a7bb91413 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:22.097522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:22.098107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:22.098382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:22.098650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:22.100355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:22.109513 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:22.110907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:22.119261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1167/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:23.720204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100ca7c-08fe-4557-857c-a5459fdde7b7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:23.730692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100ca7c-08fe-4557-857c-a5459fdde7b7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:23.734307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:23.734997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:23.735271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:23.735511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:23.737527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:23.814298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:23.815892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:23.827107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1174/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:28.852368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:28.864969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6028de0-5364-45e8-831d-d6f0b430f0c9 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:28.870929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:28.871535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:28.871878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:28.872154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:28.873887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:28.887435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:28.889086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:28.900386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1168/2342] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:28.918548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:28.920099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:28.945071 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6977d3b-ca76-4045-8008-7963a3321cef service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:28.949711 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:28.950350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:28.950631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:28.950902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] PATH_INFO: `/v3/users/703464e4a9524efaa8fdfc1870158cbb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:28.952086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=703464e4a9524efaa8fdfc1870158cbb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:28.953996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.003955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '703464e4a9524efaa8fdfc1870158cbb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.012406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '703464e4a9524efaa8fdfc1870158cbb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.024924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '703464e4a9524efaa8fdfc1870158cbb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.025524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '703464e4a9524efaa8fdfc1870158cbb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.032501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-73e5515d-cd19-413c-b74f-3ce4f675d633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '703464e4a9524efaa8fdfc1870158cbb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.039500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1175/2343] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:17:28 2022] DELETE /identity/v3/users/703464e4a9524efaa8fdfc1870158cbb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:29.212394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.226535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaec56-4d53-4cca-9ce7-3303573f9270 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:29.285585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:29.286206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.286514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.286912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] PATH_INFO: `/v3/projects/91ee5ad0b4064b9d8a038980786072cb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.288666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ee5ad0b4064b9d8a038980786072cb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:29.290564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.338216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ee5ad0b4064b9d8a038980786072cb', 'request_id': 'req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.343765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ee5ad0b4064b9d8a038980786072cb', 'request_id': 'req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.352043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ee5ad0b4064b9d8a038980786072cb', 'request_id': 'req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.363306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] The token cache is being invalidate because project 91ee5ad0b4064b9d8a038980786072cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:17:29.363622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.364280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7d7d63ff-8e81-4c6d-b137-ea10773d9b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:29.366008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1169/2344] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:17:29 2022] DELETE /identity/v3/projects/91ee5ad0b4064b9d8a038980786072cb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:29.371151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1345f-b783-44dd-92a6-f96344effa7e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.371444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1345f-b783-44dd-92a6-f96344effa7e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.371725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1345f-b783-44dd-92a6-f96344effa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.391322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4dc1345f-b783-44dd-92a6-f96344effa7e None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:29.392559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4dc1345f-b783-44dd-92a6-f96344effa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:29.399005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:29.455329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1176/2345] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:29.463651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256bcf9-6056-45e2-b34a-b5a1913421b6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.463966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256bcf9-6056-45e2-b34a-b5a1913421b6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.464236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256bcf9-6056-45e2-b34a-b5a1913421b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.483992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4256bcf9-6056-45e2-b34a-b5a1913421b6 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:29.485175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4256bcf9-6056-45e2-b34a-b5a1913421b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:29.493288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1170/2346] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:29.497310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1345f-b783-44dd-92a6-f96344effa7e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.506815 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1345f-b783-44dd-92a6-f96344effa7e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:29.511309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:29.511930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.512211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.512478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.513108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:17:29.513405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:29.514955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.526640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1177/2347] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:17:29.530590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256bcf9-6056-45e2-b34a-b5a1913421b6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.540415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256bcf9-6056-45e2-b34a-b5a1913421b6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:29.545231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:29.545827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.546111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.546386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.547021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:29.548935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.575014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1171/2348] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:17:29 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:17:29.579154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.589543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352640c9-70eb-4a8b-9529-5f01130442bd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:29.644083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:29.644683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.644989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.645272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.645895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:29.648451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.671885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1178/2349] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:17:29 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:17:29.675946 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.686071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0daf-c464-47e8-aedd-b19807fb40fc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:29.690621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:29.691207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.691484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.691775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.692414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:29.694588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.704958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1172/2350] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:29.709863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.719408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cfd51-fb8c-40d5-a398-b18193c2b605 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:29.724191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef4304-1417-4201-a572-11a812131be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:29.724806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef4304-1417-4201-a572-11a812131be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.725105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef4304-1417-4201-a572-11a812131be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.725375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef4304-1417-4201-a572-11a812131be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.726008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef4304-1417-4201-a572-11a812131be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:29.727587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef4304-1417-4201-a572-11a812131be7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.737968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1179/2351] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:29.741894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.751619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af653af-0074-46ea-8fd2-908538f241f7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:29.756634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:29.757262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.757536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.757806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] PATH_INFO: `/v3/projects/ff3cfc7a0545454eb53bae1dc4086900/users/717cbcb67fe34b68b7d4796c07531a3e/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.772270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3cfc7a0545454eb53bae1dc4086900, user_id=717cbcb67fe34b68b7d4796c07531a3e, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:29.774950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:29.785437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:29.790936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1173/2352] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:17:29 2022] PUT /identity/v3/projects/ff3cfc7a0545454eb53bae1dc4086900/users/717cbcb67fe34b68b7d4796c07531a3e/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:29.802293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:29.802583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:29.802858 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:29.830925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] MFA Rules not processed for user `717cbcb67fe34b68b7d4796c07531a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:29.832212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:29.902050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1180/2353] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:29.913013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:29.914844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:30.002407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7d69c-c6e3-49cb-846f-6d3caaca23f9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:30.010617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:30.011632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:30.012009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:30.012375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:30.023839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:30.025544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:30.041625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1174/2354] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:31.941388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:31.942918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:31.961788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383b903-ee14-4648-9c9c-747fa9c67e7c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:31.967773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a4736-4dad-40b0-b7e8-7dce098d1b8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:31.968753 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4736-4dad-40b0-b7e8-7dce098d1b8f service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:31.969141 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4736-4dad-40b0-b7e8-7dce098d1b8f service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:31.969523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4736-4dad-40b0-b7e8-7dce098d1b8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:31.971517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c2a4736-4dad-40b0-b7e8-7dce098d1b8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:31.981249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:31.985091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a4736-4dad-40b0-b7e8-7dce098d1b8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:31.987009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a4736-4dad-40b0-b7e8-7dce098d1b8f service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:32.000264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1181/2355] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:32.708479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:32.767697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a94c-0aa5-48fc-a554-97b354adc3f3 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:32.772874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:32.773500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:32.773803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:32.774071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] PATH_INFO: `/v3/users/717cbcb67fe34b68b7d4796c07531a3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:32.775251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=717cbcb67fe34b68b7d4796c07531a3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:32.777235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:32.823696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717cbcb67fe34b68b7d4796c07531a3e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:32.824374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717cbcb67fe34b68b7d4796c07531a3e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:32.832994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717cbcb67fe34b68b7d4796c07531a3e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:32.846427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717cbcb67fe34b68b7d4796c07531a3e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:32.847097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717cbcb67fe34b68b7d4796c07531a3e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:32.859916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1175/2356] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:17:32 2022] DELETE /identity/v3/users/717cbcb67fe34b68b7d4796c07531a3e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:32.864591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:32.864962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:32.865209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:32.885815 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:32.887122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:32.948970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1182/2357] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:32.959516 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:32.960963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:32.967562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:33.020258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536ad95-ccb3-4e41-879e-a4dc666a0e10 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.024466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd89c0-4cfb-4df2-8e84-095432df8a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.025060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd89c0-4cfb-4df2-8e84-095432df8a7e service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.025359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd89c0-4cfb-4df2-8e84-095432df8a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.025624 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd89c0-4cfb-4df2-8e84-095432df8a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.034246 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd89c0-4cfb-4df2-8e84-095432df8a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.035653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd89c0-4cfb-4df2-8e84-095432df8a7e service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.045044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1176/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:33.307104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.308609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:33.321669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ff694-baa9-48a9-8693-38135ac74550 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.326281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.326927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.327196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.327469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] PATH_INFO: `/v3/projects/ff3cfc7a0545454eb53bae1dc4086900` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.328616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3cfc7a0545454eb53bae1dc4086900)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.330512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.376803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3cfc7a0545454eb53bae1dc4086900', 'request_id': 'req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:33.383145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3cfc7a0545454eb53bae1dc4086900', 'request_id': 'req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:33.390202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3cfc7a0545454eb53bae1dc4086900', 'request_id': 'req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:33.401659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] The token cache is being invalidate because project ff3cfc7a0545454eb53bae1dc4086900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:17:33.401954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:33.402591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d8741e1-9c08-4e52-bd43-3ca3707f6a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:33.404202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1183/2359] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:17:33 2022] DELETE /identity/v3/projects/ff3cfc7a0545454eb53bae1dc4086900 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:33.409271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62123a65-d5f8-46b8-bb6e-cd5e3c11c899 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.409728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62123a65-d5f8-46b8-bb6e-cd5e3c11c899 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.410102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62123a65-d5f8-46b8-bb6e-cd5e3c11c899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.430812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-62123a65-d5f8-46b8-bb6e-cd5e3c11c899 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:33.432059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-62123a65-d5f8-46b8-bb6e-cd5e3c11c899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:33.493005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1177/2360] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:33.500635 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85131f2-c5bf-4cb7-b939-d3f6eb4beced None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.501111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85131f2-c5bf-4cb7-b939-d3f6eb4beced None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.501414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85131f2-c5bf-4cb7-b939-d3f6eb4beced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.520827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d85131f2-c5bf-4cb7-b939-d3f6eb4beced None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:33.522056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d85131f2-c5bf-4cb7-b939-d3f6eb4beced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:33.530036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1184/2361] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:17:33.534357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62123a65-d5f8-46b8-bb6e-cd5e3c11c899 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.545173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62123a65-d5f8-46b8-bb6e-cd5e3c11c899 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.550032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.550709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.551059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.551335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.552014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:17:33.552314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.554520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.570541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1178/2362] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:17:33.574796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85131f2-c5bf-4cb7-b939-d3f6eb4beced None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.584912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85131f2-c5bf-4cb7-b939-d3f6eb4beced None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.589105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.589703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.589981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.590253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.590889 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.592810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.622545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1185/2363] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:17:33 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:33.626925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.637860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364ad35-f8fe-468f-850f-4e13f7e25b51 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.694529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.695128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.695405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.695675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.696326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.698223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.719317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1179/2364] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:17:33 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:33.723468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.734242 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e01f12-b832-4165-bada-a99f3e68cfb2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.738903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.739493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.739793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.740681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.741311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.742932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.753770 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1186/2365] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:33.757728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.767561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc867-d688-4197-bdfc-24fa506250a6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.772796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f663c68-3df5-43c2-84ac-44f79246203c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.773385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663c68-3df5-43c2-84ac-44f79246203c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.773666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663c68-3df5-43c2-84ac-44f79246203c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.773935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663c68-3df5-43c2-84ac-44f79246203c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.774565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f663c68-3df5-43c2-84ac-44f79246203c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.776212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f663c68-3df5-43c2-84ac-44f79246203c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.786586 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1180/2366] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:33.791389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.801786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c05f0-5b84-4779-8362-989392e4b87c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:33.806486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:33.807208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.807563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.807931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] PATH_INFO: `/v3/projects/bd8a90979a2c4ee7bf38e744e8fab05d/users/aefebeb32a814fc7b98c7dfe96f44823/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.823322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd8a90979a2c4ee7bf38e744e8fab05d, user_id=aefebeb32a814fc7b98c7dfe96f44823, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:33.826165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:33.839690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1187/2367] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:17:33 2022] PUT /identity/v3/projects/bd8a90979a2c4ee7bf38e744e8fab05d/users/aefebeb32a814fc7b98c7dfe96f44823/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:33.850956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1002e-ba24-4b5c-a1ad-c0bf77b0f437 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:33.851248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1002e-ba24-4b5c-a1ad-c0bf77b0f437 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:33.851518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1002e-ba24-4b5c-a1ad-c0bf77b0f437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:33.878440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:33.881864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-afd1002e-ba24-4b5c-a1ad-c0bf77b0f437 None None] MFA Rules not processed for user `aefebeb32a814fc7b98c7dfe96f44823`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:33.888149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-afd1002e-ba24-4b5c-a1ad-c0bf77b0f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:33.959532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1181/2368] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:33 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:33.964121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:33.978828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:34.033963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e3bc-1715-4520-911d-25c73c378ab3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.040507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.041440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.041919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.042193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.042884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.044878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.077255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1188/2369] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:17:33 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:34.081653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1002e-ba24-4b5c-a1ad-c0bf77b0f437 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:34.093893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1002e-ba24-4b5c-a1ad-c0bf77b0f437 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.169267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.169977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.170286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.170560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.171170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.173370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.213341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1182/2370] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:17:34 2022] POST /identity/v3/users => generated 309 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:34.214924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:34.226305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b137-a723-4d99-9b9e-34116e97d0be admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.231254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.231902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.232188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.232527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.233177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.234907 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.245945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1189/2371] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:34.250459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:34.261035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0813a-dd28-4861-9835-c342e0a2de8e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.265665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.266315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.266591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.266860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] PATH_INFO: `/v3/projects/dde2a7caa4e54fba94cd5113c7d60fba/users/d5452f56ff9c4bbd9e91993faa7e2d00/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.282449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde2a7caa4e54fba94cd5113c7d60fba, user_id=d5452f56ff9c4bbd9e91993faa7e2d00, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.285142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.304458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1183/2372] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:17:34 2022] PUT /identity/v3/projects/dde2a7caa4e54fba94cd5113c7d60fba/users/d5452f56ff9c4bbd9e91993faa7e2d00/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:34.308451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:34.318410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cae6da-fff3-4abe-8f82-71bc81fb207a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.374036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c558b02-40be-4712-a721-aa76c54da14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.374756 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558b02-40be-4712-a721-aa76c54da14f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.375035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558b02-40be-4712-a721-aa76c54da14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.375307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558b02-40be-4712-a721-aa76c54da14f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.375965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558b02-40be-4712-a721-aa76c54da14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.377538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558b02-40be-4712-a721-aa76c54da14f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.387126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1190/2373] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:34.391222 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:34.401446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55c817-4279-4039-82f5-e5a69ab69376 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.406013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.406650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.406930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.407201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] PATH_INFO: `/v3/projects/dde2a7caa4e54fba94cd5113c7d60fba/users/d5452f56ff9c4bbd9e91993faa7e2d00/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.409110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde2a7caa4e54fba94cd5113c7d60fba, user_id=d5452f56ff9c4bbd9e91993faa7e2d00, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.411721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.422391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1184/2374] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:17:34 2022] PUT /identity/v3/projects/dde2a7caa4e54fba94cd5113c7d60fba/users/d5452f56ff9c4bbd9e91993faa7e2d00/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:34.434286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63685-b463-497e-97c3-9808090978db None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.434575 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63685-b463-497e-97c3-9808090978db None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.434850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63685-b463-497e-97c3-9808090978db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.461714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e1f63685-b463-497e-97c3-9808090978db None None] MFA Rules not processed for user `d5452f56ff9c4bbd9e91993faa7e2d00`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:34.462917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e1f63685-b463-497e-97c3-9808090978db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:34.529252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1191/2375] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:34.540532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:34.542509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:34.633491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f283e6a-9ba6-436b-aa18-abcc7f977f67 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.638858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b2f3b-659f-4a32-bcef-a9b213cb4c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.639783 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2f3b-659f-4a32-bcef-a9b213cb4c40 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.640217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2f3b-659f-4a32-bcef-a9b213cb4c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.640638 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2f3b-659f-4a32-bcef-a9b213cb4c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.654382 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b2f3b-659f-4a32-bcef-a9b213cb4c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.656517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b2f3b-659f-4a32-bcef-a9b213cb4c40 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.660378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:34.672005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1185/2376] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:34 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:34.817630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f63685-b463-497e-97c3-9808090978db None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:34.827682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:34.883140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f63685-b463-497e-97c3-9808090978db None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:34.887542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:34.888202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.888479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.888748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] PATH_INFO: `/v3/users/aefebeb32a814fc7b98c7dfe96f44823` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:34.889842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefebeb32a814fc7b98c7dfe96f44823)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:34.891729 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:34.940136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefebeb32a814fc7b98c7dfe96f44823'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:34.948347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefebeb32a814fc7b98c7dfe96f44823'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:34.962230 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefebeb32a814fc7b98c7dfe96f44823'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:34.962933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefebeb32a814fc7b98c7dfe96f44823'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:34.970033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefebeb32a814fc7b98c7dfe96f44823'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:34.979332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1192/2377] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:17:34 2022] DELETE /identity/v3/users/aefebeb32a814fc7b98c7dfe96f44823 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:34.981526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9588f75-59af-4f84-bb92-3c4dee8ece1b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:34.981974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9588f75-59af-4f84-bb92-3c4dee8ece1b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:34.982090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9588f75-59af-4f84-bb92-3c4dee8ece1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:35.001462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c9588f75-59af-4f84-bb92-3c4dee8ece1b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:35.002674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c9588f75-59af-4f84-bb92-3c4dee8ece1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:35.065963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1186/2378] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:35.076620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:35.078335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:35.108391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:35.108684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:35.109013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:35.128462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:35.129754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:35.140704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587903c-e8d1-47c0-931e-6078b9eb20f5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:35.144282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:35.144886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:35.145167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:35.145440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:35.155864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:35.157285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:35.166903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1193/2379] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:35.185365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1187/2380] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:17:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:17:35.241335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:35.243262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:35.286122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:35.330465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ebc14-91bf-4940-bc31-15b3c8635bc0 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:35.336878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:35.337660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:35.338118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:35.338572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:35.347628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:35.349280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:35.357376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1194/2381] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:17:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:17:35.575070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:35.576574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:35.603951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ca747-dc69-40f5-a91a-968e8a39afc6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:35.608781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:35.609433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:35.609715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:35.609994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] PATH_INFO: `/v3/projects/bd8a90979a2c4ee7bf38e744e8fab05d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:35.611133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd8a90979a2c4ee7bf38e744e8fab05d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:35.613163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:35.699181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd8a90979a2c4ee7bf38e744e8fab05d', 'request_id': 'req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:35.712955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd8a90979a2c4ee7bf38e744e8fab05d', 'request_id': 'req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:35.727321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd8a90979a2c4ee7bf38e744e8fab05d', 'request_id': 'req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:35.744085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] The token cache is being invalidate because project bd8a90979a2c4ee7bf38e744e8fab05d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:17:35.744474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:35.745217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:35.747110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1188/2382] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:17:35 2022] DELETE /identity/v3/projects/bd8a90979a2c4ee7bf38e744e8fab05d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:35.750971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:35.764848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fbe6d-6e08-440c-a765-2ff9c575cd53 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:35.932040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:35.934010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:35.935064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:35.935352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] PATH_INFO: `/v3/users/d5452f56ff9c4bbd9e91993faa7e2d00` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:35.936673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5452f56ff9c4bbd9e91993faa7e2d00)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:35.939030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:36.023670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5452f56ff9c4bbd9e91993faa7e2d00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.064393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5452f56ff9c4bbd9e91993faa7e2d00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.134433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5452f56ff9c4bbd9e91993faa7e2d00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.135207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5452f56ff9c4bbd9e91993faa7e2d00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.153286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-256917e8-ab68-41e5-a265-30722edbb72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5452f56ff9c4bbd9e91993faa7e2d00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.182464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1195/2383] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:17:35 2022] DELETE /identity/v3/users/d5452f56ff9c4bbd9e91993faa7e2d00 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:17:36.734543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:36.737656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:36.780328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196cc1-6866-4b4c-b6e3-4ad8c7513e99 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:36.812795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:36.856396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:36.856947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:36.857460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:36.857755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] PATH_INFO: `/v3/projects/dde2a7caa4e54fba94cd5113c7d60fba` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:36.858879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde2a7caa4e54fba94cd5113c7d60fba)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:36.861018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:36.925682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde2a7caa4e54fba94cd5113c7d60fba', 'request_id': 'req-a1b4a6bd-2d38-49e5-93af-4690622781d5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.932792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde2a7caa4e54fba94cd5113c7d60fba', 'request_id': 'req-a1b4a6bd-2d38-49e5-93af-4690622781d5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.946888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde2a7caa4e54fba94cd5113c7d60fba', 'request_id': 'req-a1b4a6bd-2d38-49e5-93af-4690622781d5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.959670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] The token cache is being invalidate because project dde2a7caa4e54fba94cd5113c7d60fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:17:36.959994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.960653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a1b4a6bd-2d38-49e5-93af-4690622781d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:17:36.962341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1189/2384] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:17:36 2022] DELETE /identity/v3/projects/dde2a7caa4e54fba94cd5113c7d60fba => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:36.969663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fd892-88f1-4e69-9430-d04840c196ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:36.969663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fd892-88f1-4e69-9430-d04840c196ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:36.969663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fd892-88f1-4e69-9430-d04840c196ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:36.989438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-de2fd892-88f1-4e69-9430-d04840c196ed None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:36.991272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de2fd892-88f1-4e69-9430-d04840c196ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:37.045832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:37.093192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1196/2385] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:37.102001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc9e81-9ae1-4c25-8f29-47b56cba1075 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.102309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc9e81-9ae1-4c25-8f29-47b56cba1075 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.102580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc9e81-9ae1-4c25-8f29-47b56cba1075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.128697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0cbc9e81-9ae1-4c25-8f29-47b56cba1075 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:37.130160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0cbc9e81-9ae1-4c25-8f29-47b56cba1075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:37.152695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1190/2386] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:37.153955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fd892-88f1-4e69-9430-d04840c196ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:37.191813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fd892-88f1-4e69-9430-d04840c196ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:37.198301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:37.199174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.199628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.200355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.201224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:17:37.201690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:37.203943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:37.245692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1197/2387] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:17:37.249786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc9e81-9ae1-4c25-8f29-47b56cba1075 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:37.267282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc9e81-9ae1-4c25-8f29-47b56cba1075 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:37.282446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:37.283810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.284258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.284383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.285089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:37.287089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:37.351474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1191/2388] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:17:37 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:37.355422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:37.372007 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc3f8-7d7f-45b2-b9f9-c82a2233e666 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:37.505198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:37.505884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.506153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.506422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.507075 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:37.509052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:37.563404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:37.588914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1198/2389] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:17:37 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:37.592898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92aceb-eb2d-4f45-a5ef-128412ae6e16 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:37.598619 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:37.599587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.599987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.600366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.601166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:37.602879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:37.618472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:37.624004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1192/2390] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:37.630138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25d270-5340-46d8-a20a-010166cc84b7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:37.636131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8daf7-a0d6-4e72-8889-16214a87ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:37.636848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8daf7-a0d6-4e72-8889-16214a87ef73 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.637234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8daf7-a0d6-4e72-8889-16214a87ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.637646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8daf7-a0d6-4e72-8889-16214a87ef73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.638391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8daf7-a0d6-4e72-8889-16214a87ef73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:37.640116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8daf7-a0d6-4e72-8889-16214a87ef73 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:37.659469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1199/2391] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:17:37.664050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:37.675633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09345dd3-dd85-4d94-bfc0-cdd0a64feda1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:37.681270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:37.682434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.682434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.682904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] PATH_INFO: `/v3/projects/b9fae9787c114bfa8bfda174b0360116/users/1c402465c75b4da295770389f159f175/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.701379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fae9787c114bfa8bfda174b0360116, user_id=1c402465c75b4da295770389f159f175, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:37.704376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:37.740450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798521a-db59-43f1-b86d-efdf0859718f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:37.740878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798521a-db59-43f1-b86d-efdf0859718f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:37.741267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798521a-db59-43f1-b86d-efdf0859718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:37.744681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1193/2392] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:17:37 2022] PUT /identity/v3/projects/b9fae9787c114bfa8bfda174b0360116/users/1c402465c75b4da295770389f159f175/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:37.800955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f798521a-db59-43f1-b86d-efdf0859718f None None] MFA Rules not processed for user `1c402465c75b4da295770389f159f175`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:37.802443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f798521a-db59-43f1-b86d-efdf0859718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:37.929775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:37.933108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1200/2393] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:38.007458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:38.022922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e74cc-8d88-4434-9fd4-d4bb3eb46444 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:38.033436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:38.034112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.034394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.034670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.035341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:38.038134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:38.082161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1194/2394] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:17:37 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:38.086486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798521a-db59-43f1-b86d-efdf0859718f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:38.106191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798521a-db59-43f1-b86d-efdf0859718f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:38.167048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:38.213643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:38.214190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.214477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.214751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.215436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:38.217411 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:38.246775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1201/2395] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:17:38 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:17:38.250508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:38.270774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d38b8b-f855-4381-983c-d67a0a75cc87 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:38.284770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:38.285869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.286289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.286750 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.287871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:38.289812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:38.311950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1195/2396] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:38.313896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:38.334186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e01a-2d47-4680-8101-06096c34e059 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:38.342318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:38.343343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.343725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.344135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] PATH_INFO: `/v3/projects/52316c1427314b87af577338863e349e/users/0090993e314849c09853f9f00953f0a8/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.363318 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=52316c1427314b87af577338863e349e, user_id=0090993e314849c09853f9f00953f0a8, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:38.367434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:38.397507 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1202/2397] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:17:38 2022] PUT /identity/v3/projects/52316c1427314b87af577338863e349e/users/0090993e314849c09853f9f00953f0a8/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:38.399378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:38.411920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c9f4b-a747-4479-bba6-020cf58278d3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:38.506293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef4eac-81cf-495f-b6b6-93931bf192b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:38.507169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef4eac-81cf-495f-b6b6-93931bf192b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.507536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef4eac-81cf-495f-b6b6-93931bf192b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.507959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef4eac-81cf-495f-b6b6-93931bf192b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.508752 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef4eac-81cf-495f-b6b6-93931bf192b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:38.510883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef4eac-81cf-495f-b6b6-93931bf192b2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:38.525729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1196/2398] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:17:38.526829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:38.539218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192d38-dcb4-4a7f-8f13-9bb0899da56d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:38.544418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:38.545077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.545355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.545626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] PATH_INFO: `/v3/projects/52316c1427314b87af577338863e349e/users/0090993e314849c09853f9f00953f0a8/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.549151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] RBAC: Authorizing `identity:create_grant(project_id=52316c1427314b87af577338863e349e, user_id=0090993e314849c09853f9f00953f0a8, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:38.551958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:38.567765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1203/2399] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:17:38 2022] PUT /identity/v3/projects/52316c1427314b87af577338863e349e/users/0090993e314849c09853f9f00953f0a8/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:17:38.581320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aacee-9a8d-415d-992f-596bff8b14d6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.581601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aacee-9a8d-415d-992f-596bff8b14d6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.581866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aacee-9a8d-415d-992f-596bff8b14d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.632266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f54aacee-9a8d-415d-992f-596bff8b14d6 None None] MFA Rules not processed for user `0090993e314849c09853f9f00953f0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:38.633308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f54aacee-9a8d-415d-992f-596bff8b14d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:38.776012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1197/2400] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:17:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:38.785122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:38.787223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:38.899425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514763-a6f5-4987-bfff-09acceb0170d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:38.903316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79be67f-9771-46cc-85c9-18660d804444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:38.903992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79be67f-9771-46cc-85c9-18660d804444 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:38.904271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79be67f-9771-46cc-85c9-18660d804444 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:38.904546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79be67f-9771-46cc-85c9-18660d804444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:38.918152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79be67f-9771-46cc-85c9-18660d804444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:38.919902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79be67f-9771-46cc-85c9-18660d804444 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:38.931694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1204/2401] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:39.045627 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aacee-9a8d-415d-992f-596bff8b14d6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:39.063458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aacee-9a8d-415d-992f-596bff8b14d6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:39.080232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a460091-a81f-4e4c-afe9-48d0300cd953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:39.080966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a460091-a81f-4e4c-afe9-48d0300cd953 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:39.081256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a460091-a81f-4e4c-afe9-48d0300cd953 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:39.081734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a460091-a81f-4e4c-afe9-48d0300cd953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:39.139552 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:39.157640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a460091-a81f-4e4c-afe9-48d0300cd953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:39.159213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a460091-a81f-4e4c-afe9-48d0300cd953 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:39.170567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1198/2402] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:48.925962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:49.166682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:52.975901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:52.976341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:52.976791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:52.996530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:52.997634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:53.072522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1205/2403] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 6 01:17:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:53.076499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e7820-cbfa-4edb-ad2f-b04212c925ef None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.077154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e7820-cbfa-4edb-ad2f-b04212c925ef None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.077154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e7820-cbfa-4edb-ad2f-b04212c925ef None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.078483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1199/2404] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:17:53.081238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:53.082652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:53.084808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57037-a2ee-4234-a19d-1e23ef677f3a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.085275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57037-a2ee-4234-a19d-1e23ef677f3a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.085689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57037-a2ee-4234-a19d-1e23ef677f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.107833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bfe57037-a2ee-4234-a19d-1e23ef677f3a None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:53.109108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bfe57037-a2ee-4234-a19d-1e23ef677f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:53.172315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b685ebc-76c2-43ca-b000-e17e2f438eff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:53.178358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:53.179102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.179850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.179850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.191897 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:53.194105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:53.206813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1206/2405] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:17:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:17:53.213012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1200/2406] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:17:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:17:53.225600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:53.227017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:53.296077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860d949-583c-4180-b017-3b61d1774b9e service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:53.300214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c5f43-61d2-4ae6-8b8f-5552d0725ba6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10bbe5bd5324482b9a74836a552a077d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:53.300954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c5f43-61d2-4ae6-8b8f-5552d0725ba6 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.301353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c5f43-61d2-4ae6-8b8f-5552d0725ba6 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.301662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c5f43-61d2-4ae6-8b8f-5552d0725ba6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.313480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c5f43-61d2-4ae6-8b8f-5552d0725ba6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:53.314986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c5f43-61d2-4ae6-8b8f-5552d0725ba6 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:53.329559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1207/2407] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:53.400164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433891f-5778-49a3-a70d-5e125dd6281f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.400648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433891f-5778-49a3-a70d-5e125dd6281f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.400970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433891f-5778-49a3-a70d-5e125dd6281f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.402673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1201/2408] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:17:53.407614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668e42-6f92-4d9d-ba76-185a39ab057f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.408047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668e42-6f92-4d9d-ba76-185a39ab057f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.408442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668e42-6f92-4d9d-ba76-185a39ab057f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.430485 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a668e42-6f92-4d9d-ba76-185a39ab057f None None] MFA Rules not processed for user `bc47f8d295ed481a87b4fb0d405c605e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:53.431696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a668e42-6f92-4d9d-ba76-185a39ab057f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:53.474571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:17:53.494293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1208/2409] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:17:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:17:53.506156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433891f-5778-49a3-a70d-5e125dd6281f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:53.507949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a433891f-5778-49a3-a70d-5e125dd6281f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:53.522143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433891f-5778-49a3-a70d-5e125dd6281f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:53.525678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1025994-4052-4b8f-8c7c-912ced5a9663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:53.526392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1025994-4052-4b8f-8c7c-912ced5a9663 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.526675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1025994-4052-4b8f-8c7c-912ced5a9663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.526951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1025994-4052-4b8f-8c7c-912ced5a9663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.534299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1025994-4052-4b8f-8c7c-912ced5a9663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:53.535767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1025994-4052-4b8f-8c7c-912ced5a9663 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:53.544615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1202/2410] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:53.627887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c245a2-d35d-4cc6-ad53-935a86c8ffed None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.628182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c245a2-d35d-4cc6-ad53-935a86c8ffed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.628459 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c245a2-d35d-4cc6-ad53-935a86c8ffed None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.630548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1209/2411] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 6 01:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 01:17:53.636671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905d90-2a64-4797-8ce1-9664732b65a9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.636671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905d90-2a64-4797-8ce1-9664732b65a9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.636671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905d90-2a64-4797-8ce1-9664732b65a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.659494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-24905d90-2a64-4797-8ce1-9664732b65a9 None None] MFA Rules not processed for user `6b6caf1ad3784ddabda79ca0b1fc468d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:53.660789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-24905d90-2a64-4797-8ce1-9664732b65a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:53.671682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1203/2412] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 6 01:17:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:53.682938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c245a2-d35d-4cc6-ad53-935a86c8ffed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:53.694865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c245a2-d35d-4cc6-ad53-935a86c8ffed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:53.698670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:53.699338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:53.699621 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:53.699929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:53.710697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:53.712243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:53.722865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1210/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:17:55.838659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:55.839143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:55.839247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:55.858907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:17:55.860219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:55.868302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1204/2414] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 6 01:17:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:17:55.883327 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:55.895535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efe54b-65c2-4540-b6c6-81ddf963411c service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:55.903598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:55.904426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:55.905255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:55.905255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:55.916732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:55.918019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:55.928099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1211/2415] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:17:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 01:17:58.138415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:17:58.139861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:58.245007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa190c4-e74a-4dcc-b246-84b7b26a5b7d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:17:58.258486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:17:58.259209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:17:58.259555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:17:58.259935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:17:58.262934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:17:58.277212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:17:58.278920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:17:58.290570 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1205/2416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:17:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:02.496331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:02.497802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:02.587202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c391448-d69c-4289-910a-3e99e02e5f35 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:02.592253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:02.593134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:02.593577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:02.594008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] PATH_INFO: `/v3/users/18c5c95fde3646409d542da0ff274883` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:02.595315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c5c95fde3646409d542da0ff274883)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:02.597486 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:02.649245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c5c95fde3646409d542da0ff274883'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:02.661794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c5c95fde3646409d542da0ff274883'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:02.684283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c5c95fde3646409d542da0ff274883'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:02.685073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c5c95fde3646409d542da0ff274883'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:02.692928 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c5c95fde3646409d542da0ff274883'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:02.699148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1212/2417] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:02 2022] DELETE /identity/v3/users/18c5c95fde3646409d542da0ff274883 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:02.924335 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:02.935215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf1daf-f3f1-4948-a016-094d5a927c90 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:03.007907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:03.015781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:03.016831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:03.017116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:03.017392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] PATH_INFO: `/v3/projects/25d31e6f0604458f8b9a36e2de86ff1f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:03.023515 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d31e6f0604458f8b9a36e2de86ff1f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:03.023833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:03.099794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d31e6f0604458f8b9a36e2de86ff1f', 'request_id': 'req-62c55e51-a4e2-405a-b516-183b835f04c5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.109893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d31e6f0604458f8b9a36e2de86ff1f', 'request_id': 'req-62c55e51-a4e2-405a-b516-183b835f04c5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.120791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d31e6f0604458f8b9a36e2de86ff1f', 'request_id': 'req-62c55e51-a4e2-405a-b516-183b835f04c5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.136723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] The token cache is being invalidate because project 25d31e6f0604458f8b9a36e2de86ff1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:03.137363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.138483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.140469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1206/2418] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:02 2022] DELETE /identity/v3/projects/25d31e6f0604458f8b9a36e2de86ff1f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:03.144498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:03.146068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:03.163612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347aede-83e1-4582-80b9-cf3373bd9730 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:03.221665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:03.244506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:03.245248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:03.245605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:03.245951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] PATH_INFO: `/v3/users/46119bc0364843aaa93153b13d22e48d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:03.247229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=46119bc0364843aaa93153b13d22e48d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:03.249538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:03.298689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46119bc0364843aaa93153b13d22e48d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.309105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46119bc0364843aaa93153b13d22e48d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.328015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46119bc0364843aaa93153b13d22e48d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.328835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46119bc0364843aaa93153b13d22e48d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.335923 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c1f4fa6-ce54-451d-a8c3-f19c739a7d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46119bc0364843aaa93153b13d22e48d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.341783 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1213/2419] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:03 2022] DELETE /identity/v3/users/46119bc0364843aaa93153b13d22e48d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:03.529148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:03.530890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:03.566737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c55e51-a4e2-405a-b516-183b835f04c5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:03.690871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:03.692403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:03.692957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:03.693376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] PATH_INFO: `/v3/projects/5b79817953084f4da300b677a7ed4f3f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:03.694871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b79817953084f4da300b677a7ed4f3f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:03.697032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:03.756613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b79817953084f4da300b677a7ed4f3f', 'request_id': 'req-13a32c8e-3f16-4238-a8a1-297de3d50071'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.763361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b79817953084f4da300b677a7ed4f3f', 'request_id': 'req-13a32c8e-3f16-4238-a8a1-297de3d50071'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.774930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b79817953084f4da300b677a7ed4f3f', 'request_id': 'req-13a32c8e-3f16-4238-a8a1-297de3d50071'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.799781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] The token cache is being invalidate because project 5b79817953084f4da300b677a7ed4f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:03.799781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.805761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:03.808503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1207/2420] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:03 2022] DELETE /identity/v3/projects/5b79817953084f4da300b677a7ed4f3f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:13.338326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:13.786358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:16.679333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:16.679884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:16.680012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:16.701674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] MFA Rules not processed for user `0b29d618b99e42aab0c42b61adf45e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:16.703102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:16.767687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1214/2421] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 6 01:18:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:16.781757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:16.783519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:16.864070 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a32c8e-3f16-4238-a8a1-297de3d50071 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:16.869754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba7ec9-a21d-40d0-b4bf-a8ec757ace70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e26430820d684ca8b4f63e473a16b6ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:16.870363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7ec9-a21d-40d0-b4bf-a8ec757ace70 service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:16.870650 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7ec9-a21d-40d0-b4bf-a8ec757ace70 service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:16.870925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7ec9-a21d-40d0-b4bf-a8ec757ace70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:16.880133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba7ec9-a21d-40d0-b4bf-a8ec757ace70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:16.881736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba7ec9-a21d-40d0-b4bf-a8ec757ace70 service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:16.892658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1208/2422] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 6 01:18:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 01:18:20.009848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:20.012617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:20.081001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d43f4-4f80-4e84-bbb8-36c9d6812a41 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:20.085786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:20.086357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:20.086634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:20.086905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] PATH_INFO: `/v3/users/1c402465c75b4da295770389f159f175` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:20.088145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c402465c75b4da295770389f159f175)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:20.090038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:20.138598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c402465c75b4da295770389f159f175'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.145903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c402465c75b4da295770389f159f175'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.157517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c402465c75b4da295770389f159f175'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.158137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c402465c75b4da295770389f159f175'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.165558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c402465c75b4da295770389f159f175'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.171806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1215/2423] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:20 2022] DELETE /identity/v3/users/1c402465c75b4da295770389f159f175 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:20.176864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:20.177328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:20.177442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:20.196027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:20.197210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:20.256730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1209/2424] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:20.266795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:20.268425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:20.304764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:20.336361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ec2b7-7a8d-4e3a-9431-f8c4d4006946 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:20.340001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:20.340626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:20.340903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:20.341171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:20.350700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:20.352181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:20.361346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1216/2425] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:20.626295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:20.627817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:20.640476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a880f-84c9-4e36-9dfc-b104184a89de None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:20.646045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:20.646704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:20.646978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:20.647248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] PATH_INFO: `/v3/projects/b9fae9787c114bfa8bfda174b0360116` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:20.648395 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fae9787c114bfa8bfda174b0360116)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:20.650304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:20.699116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fae9787c114bfa8bfda174b0360116', 'request_id': 'req-a4362108-beef-4897-b2e1-8d29d233d809'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.704575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fae9787c114bfa8bfda174b0360116', 'request_id': 'req-a4362108-beef-4897-b2e1-8d29d233d809'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.712592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fae9787c114bfa8bfda174b0360116', 'request_id': 'req-a4362108-beef-4897-b2e1-8d29d233d809'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.723468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] The token cache is being invalidate because project b9fae9787c114bfa8bfda174b0360116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:20.723774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.724390 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.725876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1210/2426] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:20 2022] DELETE /identity/v3/projects/b9fae9787c114bfa8bfda174b0360116 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:20.729798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:20.739499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db316f26-6f35-49c2-a342-427db97dbe53 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:20.793673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:20.794287 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:20.794568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:20.794836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] PATH_INFO: `/v3/users/0090993e314849c09853f9f00953f0a8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:20.795955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0090993e314849c09853f9f00953f0a8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:20.797877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:20.841513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0090993e314849c09853f9f00953f0a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.847705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0090993e314849c09853f9f00953f0a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.859633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0090993e314849c09853f9f00953f0a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.860305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0090993e314849c09853f9f00953f0a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.867282 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0090993e314849c09853f9f00953f0a8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:20.872950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1217/2427] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:20 2022] DELETE /identity/v3/users/0090993e314849c09853f9f00953f0a8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:21.102548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.103907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:21.121130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:21.151002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.152618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:21.218524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:21.223167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362108-beef-4897-b2e1-8d29d233d809 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.240060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be1d51-facc-4fbb-8745-5d7a89badb09 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.246451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.247280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.247854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.248227 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] PATH_INFO: `/v3/projects/52316c1427314b87af577338863e349e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.249513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52316c1427314b87af577338863e349e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.251490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.309962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa9941-3a48-4671-a71d-1a1c283ca38c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.310687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa9941-3a48-4671-a71d-1a1c283ca38c service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.310916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa9941-3a48-4671-a71d-1a1c283ca38c service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.311193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa9941-3a48-4671-a71d-1a1c283ca38c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.312005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52316c1427314b87af577338863e349e', 'request_id': 'req-be16f8eb-56f8-41d7-85ff-291f21e5d638'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:21.312979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a8aa9941-3a48-4671-a71d-1a1c283ca38c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:21.318753 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52316c1427314b87af577338863e349e', 'request_id': 'req-be16f8eb-56f8-41d7-85ff-291f21e5d638'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:21.327642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52316c1427314b87af577338863e349e', 'request_id': 'req-be16f8eb-56f8-41d7-85ff-291f21e5d638'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:21.340651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] The token cache is being invalidate because project 52316c1427314b87af577338863e349e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:21.341115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:21.341866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be16f8eb-56f8-41d7-85ff-291f21e5d638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:21.343850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1218/2428] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:21 2022] DELETE /identity/v3/projects/52316c1427314b87af577338863e349e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:21.348256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526a1b-ea0c-43f0-8915-e8468e4c298e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.348617 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526a1b-ea0c-43f0-8915-e8468e4c298e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.348987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526a1b-ea0c-43f0-8915-e8468e4c298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.373244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d526a1b-ea0c-43f0-8915-e8468e4c298e None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:21.374822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d526a1b-ea0c-43f0-8915-e8468e4c298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:21.391877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa9941-3a48-4671-a71d-1a1c283ca38c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.393808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa9941-3a48-4671-a71d-1a1c283ca38c service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.407104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1211/2429] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:21.442211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1219/2430] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:21.456734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445420-5a1f-4999-b616-362479010c5d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.457110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445420-5a1f-4999-b616-362479010c5d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.457511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445420-5a1f-4999-b616-362479010c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.477970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9a445420-5a1f-4999-b616-362479010c5d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:21.479131 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9a445420-5a1f-4999-b616-362479010c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:21.487095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1212/2431] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:21.492095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d526a1b-ea0c-43f0-8915-e8468e4c298e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.503010 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d526a1b-ea0c-43f0-8915-e8468e4c298e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.508303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.508995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.509434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.509670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.510321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:18:21.510623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.512267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.524944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1220/2432] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:18:21.529584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a445420-5a1f-4999-b616-362479010c5d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.540125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a445420-5a1f-4999-b616-362479010c5d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.545278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.545933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.546299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.546579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.547225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.549242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.590962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1213/2433] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:21 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:21.591871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.603105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b097560-92bf-41f2-83af-f34d5afb8c10 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.663080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.663728 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.664033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.664360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.664994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.667199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.685733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:21.707567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.713567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1221/2434] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:21 2022] POST /identity/v3/users => generated 284 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:21.719280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4820b3-fe91-4b1a-9622-40ab054b545d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.724939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.725614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.725957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.726294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.726996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.728701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.741898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1214/2435] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:21.746026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.761746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad400e-f5af-44f3-925e-c5144d2f0343 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.766629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff36fc-2c64-4f47-8ca6-4ef907dbb655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.767237 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff36fc-2c64-4f47-8ca6-4ef907dbb655 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.767516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff36fc-2c64-4f47-8ca6-4ef907dbb655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.767814 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff36fc-2c64-4f47-8ca6-4ef907dbb655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.768458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff36fc-2c64-4f47-8ca6-4ef907dbb655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.770569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff36fc-2c64-4f47-8ca6-4ef907dbb655 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.781943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1222/2436] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:21.787098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:21.797437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb6212-9da9-4338-83c6-efd6ad83f8c3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:21.802048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:21.802693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.802973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.803246 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] PATH_INFO: `/v3/projects/8c87fc52e77f4f55bb37def052260fb3/users/d47d68936c8a4cf88f4f8754492614a0/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.806235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:21.818107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c87fc52e77f4f55bb37def052260fb3, user_id=d47d68936c8a4cf88f4f8754492614a0, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:21.820798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:21.833996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1215/2437] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:21 2022] PUT /identity/v3/projects/8c87fc52e77f4f55bb37def052260fb3/users/d47d68936c8a4cf88f4f8754492614a0/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:21.844615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec91078-c80f-4d4a-9f1e-d8b771b01191 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:21.844902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec91078-c80f-4d4a-9f1e-d8b771b01191 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:21.845177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec91078-c80f-4d4a-9f1e-d8b771b01191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:21.872421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1ec91078-c80f-4d4a-9f1e-d8b771b01191 None None] MFA Rules not processed for user `d47d68936c8a4cf88f4f8754492614a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:21.873550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ec91078-c80f-4d4a-9f1e-d8b771b01191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:21.936529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1223/2438] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:21.941476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.005591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93eb97-6db0-42db-93e7-bdbe08b48bdc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.010333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.010936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.011219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.011524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.012190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.014102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.042503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1216/2439] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:21 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:22.046669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec91078-c80f-4d4a-9f1e-d8b771b01191 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.057360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec91078-c80f-4d4a-9f1e-d8b771b01191 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.116514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.117275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.117618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.118003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.118929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.120914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.142572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1224/2440] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:22 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:22.146669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.156833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0711889-49c2-42e2-8d24-6d0ea4115d61 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.161903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.162491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.162770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.163040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.163664 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.165299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.176402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1217/2441] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:22.180373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.190167 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc60695-9639-4ad2-8240-0733c9e0116d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.194840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.195483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.195780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.196068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] PATH_INFO: `/v3/projects/755b48a02f0f4fb18f1cc9bdbdc90076/users/eded35403ad743cc96a1ce7ea7175107/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.210808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755b48a02f0f4fb18f1cc9bdbdc90076, user_id=eded35403ad743cc96a1ce7ea7175107, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.213481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.226829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1225/2442] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:22 2022] PUT /identity/v3/projects/755b48a02f0f4fb18f1cc9bdbdc90076/users/eded35403ad743cc96a1ce7ea7175107/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:22.230674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.241119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571485b-d2f1-48b1-80de-93d151eaead9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.296908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1662fb-62c0-45b0-bc56-d6783e54d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.297507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1662fb-62c0-45b0-bc56-d6783e54d3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.297787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1662fb-62c0-45b0-bc56-d6783e54d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.298059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1662fb-62c0-45b0-bc56-d6783e54d3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.298683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1662fb-62c0-45b0-bc56-d6783e54d3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.300923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1662fb-62c0-45b0-bc56-d6783e54d3e9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.310308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1218/2443] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:22.314332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.324469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1215d4-c2c3-4566-a689-a61e3da05b91 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.328964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.329598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.329875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.330225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] PATH_INFO: `/v3/projects/755b48a02f0f4fb18f1cc9bdbdc90076/users/eded35403ad743cc96a1ce7ea7175107/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.332189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755b48a02f0f4fb18f1cc9bdbdc90076, user_id=eded35403ad743cc96a1ce7ea7175107, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.334799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.344590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1226/2444] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:22 2022] PUT /identity/v3/projects/755b48a02f0f4fb18f1cc9bdbdc90076/users/eded35403ad743cc96a1ce7ea7175107/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:22.355086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.355374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.355647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.383262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] MFA Rules not processed for user `eded35403ad743cc96a1ce7ea7175107`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:22.384517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:22.427479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:22.454658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1219/2445] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:22.465935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.467623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:22.500211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:22.535220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac2c77-980f-4ef1-9c0d-7ade6a2aa4c7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.537564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.538183 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.538467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.538739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.542455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.543971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:22.548505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.549933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.559261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1227/2446] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:22.607828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7b9ac-d9b5-45eb-b986-7e81e29bea13 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.608930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:22.610330 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:22.611502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.612151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.612435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.612743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.614455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:22.621465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91962-8689-4993-8b6b-868cb9210b9b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:22.625089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:22.625674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:22.625954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:22.626225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:22.679549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.680932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.685616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:22.687267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:22.690943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1220/2447] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:18:22.697023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1228/2448] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:18:25.915939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:25.917621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:25.933308 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf6eda-58aa-4684-a479-ee9249a4a89d service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:25.938164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:25.938925 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:25.939306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:25.939680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] PATH_INFO: `/v3/users/f8f44df824414e5ebc8cfd2126e9d66f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:25.940952 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f44df824414e5ebc8cfd2126e9d66f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:25.942973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:25.996839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f44df824414e5ebc8cfd2126e9d66f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:25.997511 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f44df824414e5ebc8cfd2126e9d66f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.005535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f44df824414e5ebc8cfd2126e9d66f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.021417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f44df824414e5ebc8cfd2126e9d66f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.022036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f44df824414e5ebc8cfd2126e9d66f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.037717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1221/2449] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:25 2022] DELETE /identity/v3/users/f8f44df824414e5ebc8cfd2126e9d66f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:26.220311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.235809 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e766ee-71a4-4d01-8d48-d10dcb207c34 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.276471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.277956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:26.283330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:26.296981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.297615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.297890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.298160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] PATH_INFO: `/v3/projects/1fa73b0bfee14a9ea25f03e523078995` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.299341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa73b0bfee14a9ea25f03e523078995)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.301891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.348306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31c197-db5f-4e7c-9d0d-78376ea95b24 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.355388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa73b0bfee14a9ea25f03e523078995', 'request_id': 'req-e1d093e0-11bf-436d-bd5d-61f017c6f49d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.361168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa73b0bfee14a9ea25f03e523078995', 'request_id': 'req-e1d093e0-11bf-436d-bd5d-61f017c6f49d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.369308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa73b0bfee14a9ea25f03e523078995', 'request_id': 'req-e1d093e0-11bf-436d-bd5d-61f017c6f49d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.372472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:26.385511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] The token cache is being invalidate because project 1fa73b0bfee14a9ea25f03e523078995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:26.386223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.388041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e1d093e0-11bf-436d-bd5d-61f017c6f49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.389582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1229/2450] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:26 2022] DELETE /identity/v3/projects/1fa73b0bfee14a9ea25f03e523078995 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:26.395579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73bbad-0d74-454f-8185-e864ade5b30d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.395890 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73bbad-0d74-454f-8185-e864ade5b30d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.396192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73bbad-0d74-454f-8185-e864ade5b30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.416434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4b73bbad-0d74-454f-8185-e864ade5b30d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:26.417282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.417565 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b73bbad-0d74-454f-8185-e864ade5b30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:26.417916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.418197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.418469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] PATH_INFO: `/v3/users/d47d68936c8a4cf88f4f8754492614a0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.419603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47d68936c8a4cf88f4f8754492614a0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.421543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.425696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1230/2451] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:26.433117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5f549-8f4c-4442-8115-5c0604bc6e0f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.433403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5f549-8f4c-4442-8115-5c0604bc6e0f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.433675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5f549-8f4c-4442-8115-5c0604bc6e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.453530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-40f5f549-8f4c-4442-8115-5c0604bc6e0f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:26.454993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40f5f549-8f4c-4442-8115-5c0604bc6e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:26.464297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1231/2452] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:26.468341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5f549-8f4c-4442-8115-5c0604bc6e0f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.469678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47d68936c8a4cf88f4f8754492614a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.470319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47d68936c8a4cf88f4f8754492614a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.478215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5f549-8f4c-4442-8115-5c0604bc6e0f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.479119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47d68936c8a4cf88f4f8754492614a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.483148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.483838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.484188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.484531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.485243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:18:26.485631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.487272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.491933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47d68936c8a4cf88f4f8754492614a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.492595 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e762fcd8-6d4c-42b5-9852-d3fa29f9e216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47d68936c8a4cf88f4f8754492614a0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:26.499609 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1232/2453] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:18:26.504244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.505991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:26.510392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1222/2454] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:26 2022] DELETE /identity/v3/users/d47d68936c8a4cf88f4f8754492614a0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:26.515031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.515466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.515764 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.518649 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636ff26-f030-4b43-ad0b-922a21c6b576 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.536450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:26.537832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:26.562545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:26.586254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.586900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.587163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.587439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.588112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.590151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.600597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1223/2455] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:26.611082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.613332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:26.617949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1233/2456] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:26 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:26.621812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.674503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8775e43-ecf5-455f-9cea-16d8ed2a608f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.683667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1eda5-e748-4023-b861-1916298f0029 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.688837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.689475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.689700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.689970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.691229 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.693699 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.723219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1234/2457] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:26 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:26.724349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.733058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.733885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.734258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413239-15bf-4c10-8d86-cedd9ed9007a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.734373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.734533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.738872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.739456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.739736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.740037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.740670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.742327 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.744913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.746341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.756273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1235/2458] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:26.760344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.760873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1224/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:18:26.771036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559b8f0-bd24-408e-af24-6170a6724672 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.776904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71520683-1019-4faf-a28d-180118de2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.777766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71520683-1019-4faf-a28d-180118de2a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.778254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71520683-1019-4faf-a28d-180118de2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.778798 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71520683-1019-4faf-a28d-180118de2a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.779547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71520683-1019-4faf-a28d-180118de2a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.781387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71520683-1019-4faf-a28d-180118de2a81 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.794105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1236/2460] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:26.798527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.809581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1a2e0-67ca-4937-94ea-fb1a259f7cc9 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:26.817558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:26.818345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.819123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.819590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] PATH_INFO: `/v3/projects/4c485dab58fc4232844ddf88a3643f00/users/a09a955033a04334987567274e2fe344/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.839779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c485dab58fc4232844ddf88a3643f00, user_id=a09a955033a04334987567274e2fe344, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:26.842616 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:26.856535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1225/2461] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:26 2022] PUT /identity/v3/projects/4c485dab58fc4232844ddf88a3643f00/users/a09a955033a04334987567274e2fe344/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:26.868043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:26.868335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:26.868612 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:26.897287 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] MFA Rules not processed for user `a09a955033a04334987567274e2fe344`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:26.898668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:26.910363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:26.976261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1237/2462] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:26.987584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:26.989712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.041911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:27.048971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.072955 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:27.081681 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eeffa-859d-4db1-8955-f61e4c84b570 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:27.088211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:27.088818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.089097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.089372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.097655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:27.099230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:27.108348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1226/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:27.130136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367ccb6-08b7-467c-bd10-31eb11af78cd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:27.134383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:27.135024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.135302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.135571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] PATH_INFO: `/v3/projects/8c87fc52e77f4f55bb37def052260fb3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.136725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c87fc52e77f4f55bb37def052260fb3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:27.138603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:27.194681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c87fc52e77f4f55bb37def052260fb3', 'request_id': 'req-b09b97e9-7c90-4558-9228-aa778ad9f233'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.201227 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c87fc52e77f4f55bb37def052260fb3', 'request_id': 'req-b09b97e9-7c90-4558-9228-aa778ad9f233'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.208895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c87fc52e77f4f55bb37def052260fb3', 'request_id': 'req-b09b97e9-7c90-4558-9228-aa778ad9f233'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.225596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] The token cache is being invalidate because project 8c87fc52e77f4f55bb37def052260fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:27.225596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.225596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.226741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1238/2464] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:27 2022] DELETE /identity/v3/projects/8c87fc52e77f4f55bb37def052260fb3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:27.230567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:27.243612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81af54-716d-4202-b63c-c183bdc39177 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:27.307708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:27.308366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.308858 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.308970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] PATH_INFO: `/v3/users/eded35403ad743cc96a1ce7ea7175107` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.310017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eded35403ad743cc96a1ce7ea7175107)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:27.311944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:27.329954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:27.332407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.344588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b97e9-7c90-4558-9228-aa778ad9f233 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:27.350438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:27.351117 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.351464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.351850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] PATH_INFO: `/v3/users/a09a955033a04334987567274e2fe344` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.353045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09a955033a04334987567274e2fe344)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:27.355012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:27.364189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eded35403ad743cc96a1ce7ea7175107'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.364849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eded35403ad743cc96a1ce7ea7175107'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.372484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eded35403ad743cc96a1ce7ea7175107'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.386593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eded35403ad743cc96a1ce7ea7175107'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.387339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4507797-4232-4db5-9092-1b251882f19a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eded35403ad743cc96a1ce7ea7175107'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.397806 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:27.403512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1227/2465] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:27 2022] DELETE /identity/v3/users/eded35403ad743cc96a1ce7ea7175107 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:27.407238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09a955033a04334987567274e2fe344'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.418948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09a955033a04334987567274e2fe344'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.434795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09a955033a04334987567274e2fe344'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.435509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09a955033a04334987567274e2fe344'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.443545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09a955033a04334987567274e2fe344'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.451788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1239/2466] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:27 2022] DELETE /identity/v3/users/a09a955033a04334987567274e2fe344 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:27.454572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.454859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.455126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.475033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:27.476290 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.537133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1228/2467] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:27.547159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:27.549153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.617899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630f9e0-b9d0-48f0-8592-60096c8dac87 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:27.621581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dd875-b719-4a77-a051-e1a965f32b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:27.622196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dd875-b719-4a77-a051-e1a965f32b4e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.622615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dd875-b719-4a77-a051-e1a965f32b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.622896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dd875-b719-4a77-a051-e1a965f32b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.633059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dd875-b719-4a77-a051-e1a965f32b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:27.634479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dd875-b719-4a77-a051-e1a965f32b4e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:27.644667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1240/2468] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:27.730117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:27.731577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.744408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fc002-4542-460e-8be8-151aeee76de3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:27.749502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:27.750052 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.750318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.750609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] PATH_INFO: `/v3/projects/755b48a02f0f4fb18f1cc9bdbdc90076` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.751706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=755b48a02f0f4fb18f1cc9bdbdc90076)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:27.753658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:27.756342 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:27.814435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755b48a02f0f4fb18f1cc9bdbdc90076', 'request_id': 'req-59fe6516-087f-4a42-beb2-ab75fbefc338'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.821717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755b48a02f0f4fb18f1cc9bdbdc90076', 'request_id': 'req-59fe6516-087f-4a42-beb2-ab75fbefc338'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.829251 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755b48a02f0f4fb18f1cc9bdbdc90076', 'request_id': 'req-59fe6516-087f-4a42-beb2-ab75fbefc338'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.844041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] The token cache is being invalidate because project 755b48a02f0f4fb18f1cc9bdbdc90076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:27.844323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.844973 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-59fe6516-087f-4a42-beb2-ab75fbefc338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:27.846452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1229/2469] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:27 2022] DELETE /identity/v3/projects/755b48a02f0f4fb18f1cc9bdbdc90076 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:27.853048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.853340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.853614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.872868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:27.874089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.954407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1241/2470] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:27.962970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783472-1e05-43eb-b5cc-44d54b2288b5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:27.963339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783472-1e05-43eb-b5cc-44d54b2288b5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:27.963866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783472-1e05-43eb-b5cc-44d54b2288b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:27.974787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:27.977309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.989558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-72783472-1e05-43eb-b5cc-44d54b2288b5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:27.991121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-72783472-1e05-43eb-b5cc-44d54b2288b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:27.999477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c8998-09a6-474a-a6cb-2e00ae615670 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.001060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1230/2471] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:28.006217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783472-1e05-43eb-b5cc-44d54b2288b5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.015569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.016251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.016529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.016823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] PATH_INFO: `/v3/projects/4c485dab58fc4232844ddf88a3643f00` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.018031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783472-1e05-43eb-b5cc-44d54b2288b5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.019265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c485dab58fc4232844ddf88a3643f00)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.021267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.026906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.027629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.028008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.028447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.029237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:18:28.029725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.031429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.047177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1231/2472] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:18:28.052088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.065346 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff268848-19c3-4814-997f-d0701280674b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.077666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c485dab58fc4232844ddf88a3643f00', 'request_id': 'req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:28.080042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:28.085257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c485dab58fc4232844ddf88a3643f00', 'request_id': 'req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:28.095513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c485dab58fc4232844ddf88a3643f00', 'request_id': 'req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:28.108257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] The token cache is being invalidate because project 4c485dab58fc4232844ddf88a3643f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:28.108681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:28.109495 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c2eaa44-7bb6-4006-8472-bd015b00f3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:28.114648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1242/2473] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:27 2022] DELETE /identity/v3/projects/4c485dab58fc4232844ddf88a3643f00 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:28.116332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1ee3-92fc-4e53-8c19-b909adba536c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.116748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1ee3-92fc-4e53-8c19-b909adba536c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.117110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1ee3-92fc-4e53-8c19-b909adba536c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.131157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.131908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.132298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.132637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.133328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.135359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.138021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-df3c1ee3-92fc-4e53-8c19-b909adba536c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:28.139259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-df3c1ee3-92fc-4e53-8c19-b909adba536c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:28.148583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1243/2474] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:28.157423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bcc5a-d018-447e-9210-361711d78d77 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.157423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bcc5a-d018-447e-9210-361711d78d77 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.157682 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bcc5a-d018-447e-9210-361711d78d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.173767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1232/2475] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:28.174772 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.177871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:28.180353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-890bcc5a-d018-447e-9210-361711d78d77 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:28.181538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-890bcc5a-d018-447e-9210-361711d78d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:28.244453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:28.249231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2570f6-6e00-4024-a470-8d561109482b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.252227 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1244/2476] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:28.256208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.256957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.257097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bcc5a-d018-447e-9210-361711d78d77 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.257367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.257704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.258583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.260662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.266521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bcc5a-d018-447e-9210-361711d78d77 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.272001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.272600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.272876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.273146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.273784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:18:28.274083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.275773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.289032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1233/2477] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:28.290273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.294735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1245/2478] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:18:28.295646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.300069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02256b52-ebe1-4eb1-9ea2-e1b6b6e8f80a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.304689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.304986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273f14a-3255-4efd-8d09-569028ae5458 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.305264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.305547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.305825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.306456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.308139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.309644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.310207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.310484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.310752 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.311354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.313222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.318380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1234/2479] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:28.322165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.333197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539537c5-52cd-4df0-92ae-78606b7f545a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.338075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.338706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.338988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.339272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.339965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.341686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.357913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1235/2480] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:28.359115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.369632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39545377-8c4b-4451-8046-ae09dca5b302 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.374473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d1657-6474-451c-9042-84e8b0286523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.375162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d1657-6474-451c-9042-84e8b0286523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.375443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d1657-6474-451c-9042-84e8b0286523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.375718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d1657-6474-451c-9042-84e8b0286523 admin admin] PATH_INFO: `/v3/projects/93b6e0a478f844f5a287b7346b7dbd79/users/b444869cdf8c47c3a6970bef16cea6f9/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.392480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d1657-6474-451c-9042-84e8b0286523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b6e0a478f844f5a287b7346b7dbd79, user_id=b444869cdf8c47c3a6970bef16cea6f9, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.395801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d1657-6474-451c-9042-84e8b0286523 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.410343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1236/2481] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:28 2022] PUT /identity/v3/projects/93b6e0a478f844f5a287b7346b7dbd79/users/b444869cdf8c47c3a6970bef16cea6f9/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:28.421269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e402-e8cf-46db-a5e6-be230843e822 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.421560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e402-e8cf-46db-a5e6-be230843e822 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.421838 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e402-e8cf-46db-a5e6-be230843e822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.450974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8554e402-e8cf-46db-a5e6-be230843e822 None None] MFA Rules not processed for user `b444869cdf8c47c3a6970bef16cea6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:28.452231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8554e402-e8cf-46db-a5e6-be230843e822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:28.465431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1246/2482] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:28.466385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.530760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1237/2483] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:28.533660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a699b-b957-46b7-8286-cc090fdb46a0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.535037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554e402-e8cf-46db-a5e6-be230843e822 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.538314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.538916 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.539188 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.539454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.540224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.542348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.564733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1247/2484] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:28.569678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.579647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f345d0f-cfa1-4a73-b617-c76ee71ab0a7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.584762 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.585447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.585801 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.586129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.587614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.588623 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.600393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1248/2485] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:28.604252 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.614347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd3c62-a026-428c-813b-2a23b5c99fee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.619074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.619666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.619985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.620274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.620881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.621233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554e402-e8cf-46db-a5e6-be230843e822 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.622439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.626399 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.627006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.627315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.627590 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.629265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.631169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.636899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1249/2486] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:28.637905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.647542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91f496-5e0d-42e5-8b4b-7186f2b55be1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.653290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12746cf-8317-4a9e-912b-29bd8846092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.653953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12746cf-8317-4a9e-912b-29bd8846092d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.654232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12746cf-8317-4a9e-912b-29bd8846092d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.654501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12746cf-8317-4a9e-912b-29bd8846092d admin admin] PATH_INFO: `/v3/projects/414abc1db54d49dd89faf08dca7a5126/users/3fbc65e3e4994c359999c435bf613d85/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.657026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:28.671161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1238/2487] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:28.672091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.674018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12746cf-8317-4a9e-912b-29bd8846092d admin admin] RBAC: Authorizing `identity:create_grant(project_id=414abc1db54d49dd89faf08dca7a5126, user_id=3fbc65e3e4994c359999c435bf613d85, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.678040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12746cf-8317-4a9e-912b-29bd8846092d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.693463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1250/2488] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:28 2022] PUT /identity/v3/projects/414abc1db54d49dd89faf08dca7a5126/users/3fbc65e3e4994c359999c435bf613d85/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:28.700993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:28.703903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0afbb7-935a-482d-aa89-5d7fc1cd55b2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.704192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0afbb7-935a-482d-aa89-5d7fc1cd55b2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.704739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0afbb7-935a-482d-aa89-5d7fc1cd55b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.732450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cf0afbb7-935a-482d-aa89-5d7fc1cd55b2 None None] MFA Rules not processed for user `3fbc65e3e4994c359999c435bf613d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:28.733667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf0afbb7-935a-482d-aa89-5d7fc1cd55b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:28.752009 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b2561-f1d6-4b1f-89ba-d752ae9a61f9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.800787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1251/2489] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:28.805201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0afbb7-935a-482d-aa89-5d7fc1cd55b2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.809485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.810095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.810374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.810669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.811291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.813227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.834713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1239/2490] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/users => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:28.838716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.848224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6af2a3-bab5-46cf-8842-e357f2cfdd79 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.852868 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.853585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.853873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.854159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.854764 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.856405 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.866754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1240/2491] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:28.870621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.873587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0afbb7-935a-482d-aa89-5d7fc1cd55b2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.878784 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.879396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.879696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.879998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.880618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.882044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd45c-8da1-4d6a-94dd-f9ecb003a4d3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.882509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.886881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.887534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.887850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.888124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] PATH_INFO: `/v3/projects/eaa564d3e9304fc0933c05c7c44a88f3/users/8c0bff0066c740b292d54656f8524200/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.903682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa564d3e9304fc0933c05c7c44a88f3, user_id=8c0bff0066c740b292d54656f8524200, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.906503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.917392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1252/2492] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:28.921803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.926684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1241/2493] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:28 2022] PUT /identity/v3/projects/eaa564d3e9304fc0933c05c7c44a88f3/users/8c0bff0066c740b292d54656f8524200/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:28.930774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:28.983063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:28.987001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4f271-0e27-4a7d-b807-22f44465683f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.992407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96370326-790a-45a1-8514-11db2ae342b8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:28.993331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.993933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.994245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.994478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.995113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:28.997041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:28.997420 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:28.998005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:28.998286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:28.998559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:28.999192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.000833 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.011079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1242/2494] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:29.015263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.019656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1253/2495] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:28 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:29.023363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.025003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c28497-85ae-4c3e-a575-6795e3ea3753 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.029735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.030359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.030642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.030915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] PATH_INFO: `/v3/projects/eaa564d3e9304fc0933c05c7c44a88f3/users/8c0bff0066c740b292d54656f8524200/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.032696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffd371-5185-47d7-aec6-775824be5f8f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.032823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa564d3e9304fc0933c05c7c44a88f3, user_id=8c0bff0066c740b292d54656f8524200, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.036033 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.037833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86d584-bc1e-4fec-8762-75f79b91ad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.038402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d584-bc1e-4fec-8762-75f79b91ad69 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.038687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d584-bc1e-4fec-8762-75f79b91ad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.038951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d584-bc1e-4fec-8762-75f79b91ad69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.039574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86d584-bc1e-4fec-8762-75f79b91ad69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.041450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86d584-bc1e-4fec-8762-75f79b91ad69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.042669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:29.047576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1243/2496] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:29 2022] PUT /identity/v3/projects/eaa564d3e9304fc0933c05c7c44a88f3/users/8c0bff0066c740b292d54656f8524200/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:29.052751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1254/2497] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:29.056739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.116045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.116331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.116605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.128907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df645fde-b664-49e2-b281-0397feda5f7a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.133663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.134320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.134602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.134878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] PATH_INFO: `/v3/projects/841cbeb75ee540d19afd1ac4912f42ef/users/544a0a3b217e46b9a0699584aae4b7ec/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.143407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] MFA Rules not processed for user `8c0bff0066c740b292d54656f8524200`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:29.145307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:29.148923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=841cbeb75ee540d19afd1ac4912f42ef, user_id=544a0a3b217e46b9a0699584aae4b7ec, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.151687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.165473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1244/2498] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:29 2022] PUT /identity/v3/projects/841cbeb75ee540d19afd1ac4912f42ef/users/544a0a3b217e46b9a0699584aae4b7ec/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:29.169536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.222902 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1255/2499] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:29.234362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.235773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:29.240826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcf247-2e94-4397-84e1-1e2ec59ba83b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.245623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.246387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.246667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.246957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.247617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.249265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.260607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1245/2500] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:29.264812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.275692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0c2bc-9cdf-4426-85d7-f99bb0857276 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.282910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3cbf7-c867-495c-89db-d0634080f238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.283583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3cbf7-c867-495c-89db-d0634080f238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.283884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3cbf7-c867-495c-89db-d0634080f238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.284276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3cbf7-c867-495c-89db-d0634080f238 admin admin] PATH_INFO: `/v3/projects/841cbeb75ee540d19afd1ac4912f42ef/users/544a0a3b217e46b9a0699584aae4b7ec/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.289908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3cbf7-c867-495c-89db-d0634080f238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841cbeb75ee540d19afd1ac4912f42ef, user_id=544a0a3b217e46b9a0699584aae4b7ec, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.292841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3cbf7-c867-495c-89db-d0634080f238 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.299654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907b6b8-eb50-40a3-8cbc-b31ce594fafd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.304010 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1246/2501] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:29 2022] PUT /identity/v3/projects/841cbeb75ee540d19afd1ac4912f42ef/users/544a0a3b217e46b9a0699584aae4b7ec/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:29.315569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.316307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.316790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.350863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.351523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.351645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] MFA Rules not processed for user `544a0a3b217e46b9a0699584aae4b7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:29.351876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.352168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.352995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:29.383796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:29.414247 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:29.425314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.426842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.434341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1247/2502] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:29.436430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1256/2503] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:18:29.446064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.447487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:29.461354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eeffdb-0c7d-4f64-bfb3-48ee4da3b55d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.465181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.465780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.466055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.466324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.476617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.478071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.488429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1248/2504] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:29.511047 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.519944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330c51-78fe-4a04-bcba-1f18da6ce7e3 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.523808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.524412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.524692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.525070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.589008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.590515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.601284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1257/2505] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:29.630166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:29.638719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b901cfb-e020-4059-a932-f54f8079ffd1 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:29.642489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562b21-789a-4917-adbf-71cc67ffef9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:29.643169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562b21-789a-4917-adbf-71cc67ffef9e service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:29.643543 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562b21-789a-4917-adbf-71cc67ffef9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:29.643930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562b21-789a-4917-adbf-71cc67ffef9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:29.716904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562b21-789a-4917-adbf-71cc67ffef9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:29.725602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562b21-789a-4917-adbf-71cc67ffef9e service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:29.736268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1249/2506] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:29.968562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:30.036103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575513-e19f-4019-afc4-a51cc325d2da service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:30.041099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:30.041866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:30.041996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:30.042283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] PATH_INFO: `/v3/users/b444869cdf8c47c3a6970bef16cea6f9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:30.044208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b444869cdf8c47c3a6970bef16cea6f9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:30.046212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:30.099100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b444869cdf8c47c3a6970bef16cea6f9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.112990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b444869cdf8c47c3a6970bef16cea6f9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.125943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:30.128515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b444869cdf8c47c3a6970bef16cea6f9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.129310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b444869cdf8c47c3a6970bef16cea6f9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.138372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b444869cdf8c47c3a6970bef16cea6f9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.145556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1258/2507] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:29 2022] DELETE /identity/v3/users/b444869cdf8c47c3a6970bef16cea6f9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:30.149987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:30.150463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:30.150912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:30.172079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:30.173309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:30.230710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1250/2508] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:30.240412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:30.241850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:30.297869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c880a1-f26b-4282-a7ff-11127bbaf44a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:30.301505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:30.302094 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:30.302367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:30.302634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:30.311361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:30.312803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:30.322064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1259/2509] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:30.583538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:30.584975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:30.596866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86886ac4-e29b-462d-adf5-04ecdba159ce None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:30.601466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:30.602099 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:30.602414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:30.602647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] PATH_INFO: `/v3/projects/93b6e0a478f844f5a287b7346b7dbd79` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:30.603797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b6e0a478f844f5a287b7346b7dbd79)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:30.605728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:30.651045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b6e0a478f844f5a287b7346b7dbd79', 'request_id': 'req-8ed43110-5ee0-404e-b030-724379fdfd85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.656850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b6e0a478f844f5a287b7346b7dbd79', 'request_id': 'req-8ed43110-5ee0-404e-b030-724379fdfd85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.663174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b6e0a478f844f5a287b7346b7dbd79', 'request_id': 'req-8ed43110-5ee0-404e-b030-724379fdfd85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.671310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:30.675977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] The token cache is being invalidate because project 93b6e0a478f844f5a287b7346b7dbd79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:30.676445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.677606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.679315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1251/2510] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:30 2022] DELETE /identity/v3/projects/93b6e0a478f844f5a287b7346b7dbd79 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:30.683288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:30.693761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675ea46-c1d3-4868-ba85-6fd5f4dca503 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:30.747377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:30.748489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:30.748775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:30.749048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] PATH_INFO: `/v3/users/8c0bff0066c740b292d54656f8524200` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:30.750143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0bff0066c740b292d54656f8524200)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:30.752090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:30.797596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0bff0066c740b292d54656f8524200'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.804888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0bff0066c740b292d54656f8524200'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.817130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0bff0066c740b292d54656f8524200'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.817792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0bff0066c740b292d54656f8524200'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.824583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cbf471dd-0231-4bca-a13b-7092280e7f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0bff0066c740b292d54656f8524200'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:30.830286 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1260/2511] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:30 2022] DELETE /identity/v3/users/8c0bff0066c740b292d54656f8524200 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:31.105593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.107991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:31.124166 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed43110-5ee0-404e-b030-724379fdfd85 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.200819 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.201552 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.201893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.202236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] PATH_INFO: `/v3/projects/eaa564d3e9304fc0933c05c7c44a88f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.203484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa564d3e9304fc0933c05c7c44a88f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.205551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:31.300744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa564d3e9304fc0933c05c7c44a88f3', 'request_id': 'req-45fa264f-6b51-483a-a7a2-3e4a532ee733'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:31.308359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa564d3e9304fc0933c05c7c44a88f3', 'request_id': 'req-45fa264f-6b51-483a-a7a2-3e4a532ee733'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:31.318389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa564d3e9304fc0933c05c7c44a88f3', 'request_id': 'req-45fa264f-6b51-483a-a7a2-3e4a532ee733'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:31.331906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] The token cache is being invalidate because project eaa564d3e9304fc0933c05c7c44a88f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:31.332236 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:31.332984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-45fa264f-6b51-483a-a7a2-3e4a532ee733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:31.338859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1252/2512] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:31 2022] DELETE /identity/v3/projects/eaa564d3e9304fc0933c05c7c44a88f3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:31.340865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e16cf-7446-4a62-ad6e-45eac95fc7f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.341333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e16cf-7446-4a62-ad6e-45eac95fc7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.341840 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e16cf-7446-4a62-ad6e-45eac95fc7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.370127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:31.376255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0a4e16cf-7446-4a62-ad6e-45eac95fc7f2 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:31.378520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a4e16cf-7446-4a62-ad6e-45eac95fc7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:31.452604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1261/2513] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:31.454749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a8b39-76db-40b7-a658-bb946201493c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.455117 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a8b39-76db-40b7-a658-bb946201493c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.455468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a8b39-76db-40b7-a658-bb946201493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.477649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e38a8b39-76db-40b7-a658-bb946201493c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:31.479169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e38a8b39-76db-40b7-a658-bb946201493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:31.488940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1253/2514] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:31.495937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e16cf-7446-4a62-ad6e-45eac95fc7f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.503456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e16cf-7446-4a62-ad6e-45eac95fc7f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.510154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.510791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.511084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.511338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.512050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:18:31.512301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.513896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:31.531821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a8b39-76db-40b7-a658-bb946201493c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.531966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1262/2515] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:18:31.543067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a8b39-76db-40b7-a658-bb946201493c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.548831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.550818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.550818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.550818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.550818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.552632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:31.589988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1254/2516] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:31 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:31.594474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.605176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f78eb-ef45-45ff-9d03-ddeb9c790900 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.664127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.664873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.665164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.665440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.666364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.668294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:31.689299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1263/2517] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:31 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:31.693665 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.703774 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af7d39-4f31-4534-b9b0-d855f43ea6ff admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.708914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.709502 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.709779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.710048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.710706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.712320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:31.722557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1255/2518] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:31.726340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.736159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee96db-4567-4708-86b0-15966629da3d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.740665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5188364-64e2-4fce-b1c4-f7abc6b44167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.741251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5188364-64e2-4fce-b1c4-f7abc6b44167 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.741527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5188364-64e2-4fce-b1c4-f7abc6b44167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.741802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5188364-64e2-4fce-b1c4-f7abc6b44167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.742429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5188364-64e2-4fce-b1c4-f7abc6b44167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.744017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5188364-64e2-4fce-b1c4-f7abc6b44167 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:31.754299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1264/2519] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:31.758402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.769962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12258f9d-0cef-4743-a5d1-00c30694b1de admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.774810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.775459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.775743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.776055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] PATH_INFO: `/v3/projects/81ba9809f1a041c8ab5e8c1eff132016/users/3a8b6e7a9844461b9b4bccad161c7877/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.791153 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ba9809f1a041c8ab5e8c1eff132016, user_id=3a8b6e7a9844461b9b4bccad161c7877, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.793845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:31.806581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1256/2520] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:31 2022] PUT /identity/v3/projects/81ba9809f1a041c8ab5e8c1eff132016/users/3a8b6e7a9844461b9b4bccad161c7877/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:31.816580 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de2b2a-c227-4fe9-b1db-aa7eac25c5d7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.816892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de2b2a-c227-4fe9-b1db-aa7eac25c5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.817169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de2b2a-c227-4fe9-b1db-aa7eac25c5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.846305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-99de2b2a-c227-4fe9-b1db-aa7eac25c5d7 None None] MFA Rules not processed for user `3a8b6e7a9844461b9b4bccad161c7877`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:31.848273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99de2b2a-c227-4fe9-b1db-aa7eac25c5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:31.913388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1265/2521] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:31.918193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:31.948349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:31.981812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e653243-0f4f-4644-9cf6-8a68b3f767a6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:31.986573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:31.987245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:31.987545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:31.987888 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:31.988617 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:31.990522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.016194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1257/2522] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:31 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:18:32.020827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de2b2a-c227-4fe9-b1db-aa7eac25c5d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.026156 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:32.033013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de2b2a-c227-4fe9-b1db-aa7eac25c5d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.093304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.093909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.094189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.094462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.095122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.097042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.119084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1266/2523] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:32 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:32.123132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.133792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e834974-4386-4725-9152-9f470737abeb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.138397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.138984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.139264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.139537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.140202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.141834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.152999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1258/2524] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:32.156746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.167112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77006f3-22f3-46d1-bf1a-3826029bba20 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.171744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584968f-82da-4ef1-a2a8-cae9dcc49603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.172354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584968f-82da-4ef1-a2a8-cae9dcc49603 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.172630 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584968f-82da-4ef1-a2a8-cae9dcc49603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.172898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584968f-82da-4ef1-a2a8-cae9dcc49603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.173522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584968f-82da-4ef1-a2a8-cae9dcc49603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.175135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584968f-82da-4ef1-a2a8-cae9dcc49603 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.184823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1267/2525] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:32.188941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.199280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587e86c-7517-4466-a466-fcd6c07da16c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.203905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.204546 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.204827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.205100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] PATH_INFO: `/v3/projects/665b35f5fe9a47f983a47dc974bc123a/users/739bae233a1749f58049a98fbaebba24/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.219802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665b35f5fe9a47f983a47dc974bc123a, user_id=739bae233a1749f58049a98fbaebba24, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.222424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.234836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1259/2526] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:32 2022] PUT /identity/v3/projects/665b35f5fe9a47f983a47dc974bc123a/users/739bae233a1749f58049a98fbaebba24/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:32.246140 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969b9e-60e3-4f16-adde-a04b70fe986b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.246431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969b9e-60e3-4f16-adde-a04b70fe986b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.246702 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969b9e-60e3-4f16-adde-a04b70fe986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.274601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c8969b9e-60e3-4f16-adde-a04b70fe986b None None] MFA Rules not processed for user `739bae233a1749f58049a98fbaebba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:32.275819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8969b9e-60e3-4f16-adde-a04b70fe986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:32.338663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1268/2527] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:32.343108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.407924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426421a-ae10-4cd1-9a77-72e3228b1c09 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.412704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.413304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.413582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.413854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.414490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.416685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.444941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1260/2528] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:32 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:32.449017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969b9e-60e3-4f16-adde-a04b70fe986b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.459690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969b9e-60e3-4f16-adde-a04b70fe986b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.516266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.516882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.517165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.517439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.518067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.520063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.538490 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:32.551777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1269/2529] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:32 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:32.553168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.564618 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a3cf0-8d84-440f-8c93-19e9d25ea074 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.583042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.585805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.586105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.586388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.587051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.588886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.602881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1261/2530] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:32.607081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.618904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f818f-a91c-43df-9ce7-15ddf7d29ce6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.623803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.624466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.624748 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.625019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] PATH_INFO: `/v3/projects/8889b27c2c4d4b98a1a135d0bc896d79/users/666357314993464cad0a23ec69375be6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.642568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8889b27c2c4d4b98a1a135d0bc896d79, user_id=666357314993464cad0a23ec69375be6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.645350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.665404 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1270/2531] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:32 2022] PUT /identity/v3/projects/8889b27c2c4d4b98a1a135d0bc896d79/users/666357314993464cad0a23ec69375be6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:32.669775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.680489 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc8876-c708-4667-ba3d-4eaab2ac77a6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.725939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:32.736886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9075ef-b6fc-48cf-b81e-388d831bd439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.738120 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9075ef-b6fc-48cf-b81e-388d831bd439 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.738400 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9075ef-b6fc-48cf-b81e-388d831bd439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.738673 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9075ef-b6fc-48cf-b81e-388d831bd439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.739314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9075ef-b6fc-48cf-b81e-388d831bd439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.740962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9075ef-b6fc-48cf-b81e-388d831bd439 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.750524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1262/2532] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:32.754828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.765003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f4d77-139a-44a1-a1a7-c85d5682dac0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.769639 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.770276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.770561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.770836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] PATH_INFO: `/v3/projects/8889b27c2c4d4b98a1a135d0bc896d79/users/666357314993464cad0a23ec69375be6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.772813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8889b27c2c4d4b98a1a135d0bc896d79, user_id=666357314993464cad0a23ec69375be6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:32.775429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:32.786114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1271/2533] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:32 2022] PUT /identity/v3/projects/8889b27c2c4d4b98a1a135d0bc896d79/users/666357314993464cad0a23ec69375be6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:32.796533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129626e5-4dd5-4d20-badf-f7a62f9310cc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.796818 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129626e5-4dd5-4d20-badf-f7a62f9310cc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.797093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129626e5-4dd5-4d20-badf-f7a62f9310cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:32.824885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-129626e5-4dd5-4d20-badf-f7a62f9310cc None None] MFA Rules not processed for user `666357314993464cad0a23ec69375be6`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:32.826089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-129626e5-4dd5-4d20-badf-f7a62f9310cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:32.892124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1263/2534] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:32.904168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:32.906033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:32.962922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66415f-d48d-432f-a461-4c1a6bfdeefb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:32.966616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:32.967190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:32.967471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:32.967759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:33.023017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:33.024472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:33.033074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1272/2535] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:36.633575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129626e5-4dd5-4d20-badf-f7a62f9310cc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:36.643198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129626e5-4dd5-4d20-badf-f7a62f9310cc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:36.646737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:36.647334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:36.647615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:36.647927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:36.704339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:36.705843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:36.714449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1264/2536] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:37.227355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:37.235627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95447fd7-3a9a-44d0-a7fb-13ed0560afc6 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:37.239152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd65628-1e85-4604-92a9-a41dc5444bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:37.239735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65628-1e85-4604-92a9-a41dc5444bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:37.240039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65628-1e85-4604-92a9-a41dc5444bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:37.240305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65628-1e85-4604-92a9-a41dc5444bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:37.249235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd65628-1e85-4604-92a9-a41dc5444bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:37.250634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd65628-1e85-4604-92a9-a41dc5444bb6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:37.260641 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1273/2537] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:38.735601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:38.737122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:38.816187 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a98d85-b408-437a-810d-8df9d8d6e660 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:38.820635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:38.821257 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:38.821533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:38.821809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] PATH_INFO: `/v3/users/3fbc65e3e4994c359999c435bf613d85` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:38.822921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbc65e3e4994c359999c435bf613d85)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:38.824873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:38.837560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:38.873101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbc65e3e4994c359999c435bf613d85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:38.873740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbc65e3e4994c359999c435bf613d85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:38.881523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbc65e3e4994c359999c435bf613d85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:38.893897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbc65e3e4994c359999c435bf613d85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:38.894530 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbc65e3e4994c359999c435bf613d85'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:38.906372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1265/2538] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:38 2022] DELETE /identity/v3/users/3fbc65e3e4994c359999c435bf613d85 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:38.910822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:38.911310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:38.911422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:38.930002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:38.931190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:38.993774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1274/2539] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:39.004169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:39.005436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:39.062918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b6a4b-89a5-41e2-92c3-bba9e9d697b0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:39.067293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:39.067926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:39.068204 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:39.068475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:39.078041 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:39.079532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:39.089459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1266/2540] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:18:39.390908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:39.392406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:39.401931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:39.410693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f189fa1-96b3-4436-a1ab-b20241010900 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:39.416129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:39.416769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:39.417049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:39.417322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] PATH_INFO: `/v3/projects/414abc1db54d49dd89faf08dca7a5126` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:39.418480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=414abc1db54d49dd89faf08dca7a5126)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:39.420419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:39.477250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414abc1db54d49dd89faf08dca7a5126', 'request_id': 'req-0edf0236-2224-4983-bc1c-1baf2a16a9ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.484608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414abc1db54d49dd89faf08dca7a5126', 'request_id': 'req-0edf0236-2224-4983-bc1c-1baf2a16a9ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.498335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414abc1db54d49dd89faf08dca7a5126', 'request_id': 'req-0edf0236-2224-4983-bc1c-1baf2a16a9ba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.516422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] The token cache is being invalidate because project 414abc1db54d49dd89faf08dca7a5126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:39.516952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.518132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.519625 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1275/2541] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:39 2022] DELETE /identity/v3/projects/414abc1db54d49dd89faf08dca7a5126 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:39.525714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:39.536125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdd6b7-2111-4215-983b-3bba1f5a06ba service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:39.606566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:39.607147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:39.607426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:39.607730 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] PATH_INFO: `/v3/users/544a0a3b217e46b9a0699584aae4b7ec` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:39.608927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=544a0a3b217e46b9a0699584aae4b7ec)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:39.610897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:39.647585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:39.658302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544a0a3b217e46b9a0699584aae4b7ec'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.659484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544a0a3b217e46b9a0699584aae4b7ec'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.667209 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544a0a3b217e46b9a0699584aae4b7ec'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.680316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544a0a3b217e46b9a0699584aae4b7ec'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.680999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eb012258-92d0-4a3f-9d13-1ffc79441173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544a0a3b217e46b9a0699584aae4b7ec'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:39.693295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1267/2542] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:39 2022] DELETE /identity/v3/users/544a0a3b217e46b9a0699584aae4b7ec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:40.004939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.006600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:40.020549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf0236-2224-4983-bc1c-1baf2a16a9ba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.083591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.084478 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.084680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.084950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] PATH_INFO: `/v3/projects/841cbeb75ee540d19afd1ac4912f42ef` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.086098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=841cbeb75ee540d19afd1ac4912f42ef)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.088020 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.147534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841cbeb75ee540d19afd1ac4912f42ef', 'request_id': 'req-ec3ce985-fef4-4239-9649-f053c2792d29'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:40.152974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841cbeb75ee540d19afd1ac4912f42ef', 'request_id': 'req-ec3ce985-fef4-4239-9649-f053c2792d29'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:40.161118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841cbeb75ee540d19afd1ac4912f42ef', 'request_id': 'req-ec3ce985-fef4-4239-9649-f053c2792d29'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:40.172333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] The token cache is being invalidate because project 841cbeb75ee540d19afd1ac4912f42ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:40.172633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:40.173288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec3ce985-fef4-4239-9649-f053c2792d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:40.175452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1276/2543] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:40 2022] DELETE /identity/v3/projects/841cbeb75ee540d19afd1ac4912f42ef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:40.180573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbede8-3568-4780-ad1a-d586495bdae7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.181035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbede8-3568-4780-ad1a-d586495bdae7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.181172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbede8-3568-4780-ad1a-d586495bdae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.200640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3ebbede8-3568-4780-ad1a-d586495bdae7 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:40.201778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3ebbede8-3568-4780-ad1a-d586495bdae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:40.276132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1268/2544] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:40.285358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3101-a417-46dc-9343-3c2661118666 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.285358 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3101-a417-46dc-9343-3c2661118666 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.285677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3101-a417-46dc-9343-3c2661118666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.307425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-152b3101-a417-46dc-9343-3c2661118666 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:40.308742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-152b3101-a417-46dc-9343-3c2661118666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:40.318154 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1277/2545] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:40.322788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbede8-3568-4780-ad1a-d586495bdae7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.333268 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbede8-3568-4780-ad1a-d586495bdae7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.338136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.338621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.338897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.339170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.339962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:18:40.340289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.341938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.354739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1269/2546] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:18:40.359148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3101-a417-46dc-9343-3c2661118666 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.368885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3101-a417-46dc-9343-3c2661118666 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.373555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.374145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.374425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.374700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.375320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.377225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.387878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:40.411893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1278/2547] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:40 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:40.416487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.428766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672027d-d1ce-4519-bc5a-c9b910ba9251 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.495779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.496605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.496887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.497776 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.498609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.501124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.527594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1270/2548] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:40 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:40.531730 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.542552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a89f-aa39-4efc-8d8a-d9c3046c0a40 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.547839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.548515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.548862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.549204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.550439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.552127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.562650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1279/2549] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:40.568092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.594729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a316de3-401c-432c-813e-9eb0654d2b2f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.607974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06ba3c-56d3-4878-ab62-f951f79f75e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.608840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06ba3c-56d3-4878-ab62-f951f79f75e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.609119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06ba3c-56d3-4878-ab62-f951f79f75e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.609390 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06ba3c-56d3-4878-ab62-f951f79f75e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.610095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06ba3c-56d3-4878-ab62-f951f79f75e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.612852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06ba3c-56d3-4878-ab62-f951f79f75e4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.628477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1271/2550] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:40.629434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.640455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f07690-0754-41c9-a951-d43d517ae199 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.645714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.646375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.646679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.646957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] PATH_INFO: `/v3/projects/46b57794e0fb4534ac7a082037f79edd/users/2bd1f98efdf04a7192be4211b68d6ff6/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.662196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b57794e0fb4534ac7a082037f79edd, user_id=2bd1f98efdf04a7192be4211b68d6ff6, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.665243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.684428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1280/2551] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:40 2022] PUT /identity/v3/projects/46b57794e0fb4534ac7a082037f79edd/users/2bd1f98efdf04a7192be4211b68d6ff6/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:40.695601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84c1b-53ec-4430-95b9-e21784fcccbb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.696371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84c1b-53ec-4430-95b9-e21784fcccbb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.696662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84c1b-53ec-4430-95b9-e21784fcccbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.728392 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-40f84c1b-53ec-4430-95b9-e21784fcccbb None None] MFA Rules not processed for user `2bd1f98efdf04a7192be4211b68d6ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:40.730056 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-40f84c1b-53ec-4430-95b9-e21784fcccbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:40.736258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:40.806842 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1272/2552] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:18:40.811219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.890400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477f09c-1e99-451b-9ec1-1e3a25c75ab3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:40.895594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:40.896885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:40.897191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:40.897489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:40.898314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:40.900496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:40.944372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1281/2553] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:18:40 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:40.948425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84c1b-53ec-4430-95b9-e21784fcccbb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:40.959322 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84c1b-53ec-4430-95b9-e21784fcccbb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.014436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.014958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.015234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.015504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.016152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.018080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.039666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1273/2554] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:18:40 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:18:41.043929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.054882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf655a9-654f-42fd-aaab-de1cc55ce6f8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.059640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.060267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.060544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.060826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.061463 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.063058 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.073420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1282/2555] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:18:41.077377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.087675 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6da7e-44a6-47ef-9adf-4220c2a445c2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.092360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.093008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.093291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.093567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] PATH_INFO: `/v3/projects/8b063a33352e41959909c1604472f0c8/users/be045ae9af954a05b9d62148f4e6da1d/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.108289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b063a33352e41959909c1604472f0c8, user_id=be045ae9af954a05b9d62148f4e6da1d, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.111567 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.125518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1274/2556] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:41 2022] PUT /identity/v3/projects/8b063a33352e41959909c1604472f0c8/users/be045ae9af954a05b9d62148f4e6da1d/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:41.129492 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.139211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c62935-34c4-4f5f-8f25-e6e4a877cc57 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.202436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.203046 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.203327 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.203601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.204273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.205855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.218674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1283/2557] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:18:41.219897 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.222735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.224500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:41.230914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2db3a4-a5e8-40ea-8808-aa1f08ca1e85 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.235593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff258b0-2362-4014-bdb8-11372fc210a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.236278 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff258b0-2362-4014-bdb8-11372fc210a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.236425 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c402c-9730-49ef-90b6-75ef96605c90 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.236562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff258b0-2362-4014-bdb8-11372fc210a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.237461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff258b0-2362-4014-bdb8-11372fc210a0 admin admin] PATH_INFO: `/v3/projects/8b063a33352e41959909c1604472f0c8/users/be045ae9af954a05b9d62148f4e6da1d/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.239447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff258b0-2362-4014-bdb8-11372fc210a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b063a33352e41959909c1604472f0c8, user_id=be045ae9af954a05b9d62148f4e6da1d, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.242155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff258b0-2362-4014-bdb8-11372fc210a0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.242450 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.243066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.243346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.243619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] PATH_INFO: `/v3/users/3a8b6e7a9844461b9b4bccad161c7877` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.244782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8b6e7a9844461b9b4bccad161c7877)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.248316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.254288 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1275/2558] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:18:41 2022] PUT /identity/v3/projects/8b063a33352e41959909c1604472f0c8/users/be045ae9af954a05b9d62148f4e6da1d/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:41.265537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.265876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.266258 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.286106 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:41.301931 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] MFA Rules not processed for user `be045ae9af954a05b9d62148f4e6da1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:41.303181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:41.314024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8b6e7a9844461b9b4bccad161c7877'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.321957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8b6e7a9844461b9b4bccad161c7877'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.336550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8b6e7a9844461b9b4bccad161c7877'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.338484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8b6e7a9844461b9b4bccad161c7877'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.347500 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1998ef6e-3dc5-4ef7-b03c-310c15631375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8b6e7a9844461b9b4bccad161c7877'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.353536 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1284/2559] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:41 2022] DELETE /identity/v3/users/3a8b6e7a9844461b9b4bccad161c7877 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:18:41.357623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:41.357787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.357932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.358203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.378534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:18:41.380006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:41.384380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1276/2560] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:41.395645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.397075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:41.441955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1285/2561] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:18:41.451733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.453175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:41.460369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9085643-ee5f-4dac-afa9-1d76f01f312c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.515155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.515859 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.516092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.516368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.518224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:41.520059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b70878-d00f-43b3-8ee8-c66be498ab87 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.523736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.524378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.524765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.525100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.534634 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.536808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.547844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1286/2562] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:18:41.590905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.592647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.605720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1277/2563] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:18:41.841065 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.842622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:41.857577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60269c61-6500-4762-ad26-738155a57873 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:41.863068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:41.863697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:41.864009 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:41.864307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] PATH_INFO: `/v3/projects/81ba9809f1a041c8ab5e8c1eff132016` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:41.865422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ba9809f1a041c8ab5e8c1eff132016)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:41.867309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:41.914882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:41.944791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ba9809f1a041c8ab5e8c1eff132016', 'request_id': 'req-fef8f144-6a0d-440f-a6f1-28d4070da773'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.951380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ba9809f1a041c8ab5e8c1eff132016', 'request_id': 'req-fef8f144-6a0d-440f-a6f1-28d4070da773'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.960873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ba9809f1a041c8ab5e8c1eff132016', 'request_id': 'req-fef8f144-6a0d-440f-a6f1-28d4070da773'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.976818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] The token cache is being invalidate because project 81ba9809f1a041c8ab5e8c1eff132016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:41.977175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.977739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:41.979547 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1287/2564] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:41 2022] DELETE /identity/v3/projects/81ba9809f1a041c8ab5e8c1eff132016 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:41.983510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:41.995111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d701f2-feb9-4ce4-9822-65b2138b4f38 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:42.050622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:42.059043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:42.059795 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:42.060085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:42.060355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] PATH_INFO: `/v3/users/739bae233a1749f58049a98fbaebba24` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:42.061482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=739bae233a1749f58049a98fbaebba24)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:42.063396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:42.117796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '739bae233a1749f58049a98fbaebba24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.118446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '739bae233a1749f58049a98fbaebba24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.125942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '739bae233a1749f58049a98fbaebba24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.138933 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '739bae233a1749f58049a98fbaebba24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.139696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '739bae233a1749f58049a98fbaebba24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.152929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1278/2565] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:41 2022] DELETE /identity/v3/users/739bae233a1749f58049a98fbaebba24 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:42.448296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:42.449997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:42.463722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8f144-6a0d-440f-a6f1-28d4070da773 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:42.527153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:42.527812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:42.528092 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:42.528364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] PATH_INFO: `/v3/projects/665b35f5fe9a47f983a47dc974bc123a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:42.529613 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=665b35f5fe9a47f983a47dc974bc123a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:42.531510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:42.578893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665b35f5fe9a47f983a47dc974bc123a', 'request_id': 'req-328c7d04-5f5e-4441-a475-0de2257f9492'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.589602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665b35f5fe9a47f983a47dc974bc123a', 'request_id': 'req-328c7d04-5f5e-4441-a475-0de2257f9492'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.596997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665b35f5fe9a47f983a47dc974bc123a', 'request_id': 'req-328c7d04-5f5e-4441-a475-0de2257f9492'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.609190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] The token cache is being invalidate because project 665b35f5fe9a47f983a47dc974bc123a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:42.609487 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.610238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.611821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1288/2566] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:42 2022] DELETE /identity/v3/projects/665b35f5fe9a47f983a47dc974bc123a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:42.616054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:42.617623 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:42.631260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64188525-c95a-4da1-8211-64c4b30bb8f7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:42.685463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:42.686094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:42.686375 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:42.686648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] PATH_INFO: `/v3/users/666357314993464cad0a23ec69375be6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:42.687836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=666357314993464cad0a23ec69375be6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:42.690404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:42.733940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666357314993464cad0a23ec69375be6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.734581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666357314993464cad0a23ec69375be6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.741780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666357314993464cad0a23ec69375be6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.754028 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666357314993464cad0a23ec69375be6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.754662 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666357314993464cad0a23ec69375be6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:42.759021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:42.769984 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1279/2567] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:18:42 2022] DELETE /identity/v3/users/666357314993464cad0a23ec69375be6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:43.046857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:43.048745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:43.067351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7d04-5f5e-4441-a475-0de2257f9492 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:43.124105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:43.136376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:43.137031 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:43.137394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:43.137576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] PATH_INFO: `/v3/projects/8889b27c2c4d4b98a1a135d0bc896d79` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:43.138796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8889b27c2c4d4b98a1a135d0bc896d79)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:43.140778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:43.202742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8889b27c2c4d4b98a1a135d0bc896d79', 'request_id': 'req-d3b3197b-c83a-4484-8849-4ce6965fa6b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:43.208199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8889b27c2c4d4b98a1a135d0bc896d79', 'request_id': 'req-d3b3197b-c83a-4484-8849-4ce6965fa6b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:43.216045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8889b27c2c4d4b98a1a135d0bc896d79', 'request_id': 'req-d3b3197b-c83a-4484-8849-4ce6965fa6b9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:43.227496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] The token cache is being invalidate because project 8889b27c2c4d4b98a1a135d0bc896d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:18:43.227816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:43.228593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:18:43.230173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1289/2568] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:18:43 2022] DELETE /identity/v3/projects/8889b27c2c4d4b98a1a135d0bc896d79 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:18:46.268611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:46.270332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:46.329950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb6959-7c43-4478-9948-cfc02c381d79 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:46.378735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:46.379442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:46.379899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:46.380183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:46.383389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:46.451148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:46.452663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:46.462165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1280/2569] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:18:53.224534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:56.458134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=73 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:18:56.980600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:18:56.982068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:57.041235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3197b-c83a-4484-8849-4ce6965fa6b9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:18:57.044769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:18:57.045372 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:18:57.045645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:18:57.045909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:18:57.047917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:18:57.115667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:18:57.117409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:18:57.127197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1290/2570] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:00.004466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.007935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:00.023874 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7be63a-a2e4-46ea-a17a-ebf9a1bbca57 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.030469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.031086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.031473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.031887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] PATH_INFO: `/v3/users/e17a4e664b2c4ac9a9fea65287faad95` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.033459 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17a4e664b2c4ac9a9fea65287faad95)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.035532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.099101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17a4e664b2c4ac9a9fea65287faad95'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.099704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17a4e664b2c4ac9a9fea65287faad95'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.108659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17a4e664b2c4ac9a9fea65287faad95'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.124768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17a4e664b2c4ac9a9fea65287faad95'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.125461 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f0e0810-868e-4ecf-b2fe-53072654fe2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17a4e664b2c4ac9a9fea65287faad95'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.144219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1281/2571] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:00 2022] DELETE /identity/v3/users/e17a4e664b2c4ac9a9fea65287faad95 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:00.338066 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.351812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119a948-ade3-45a9-b34b-dd0670093464 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.413732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.414410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.414698 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.414978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] PATH_INFO: `/v3/projects/80cfec738cab491ba413e062b5a00813` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.416471 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cfec738cab491ba413e062b5a00813)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.418441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.462740 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cfec738cab491ba413e062b5a00813', 'request_id': 'req-9ed53cef-729e-42b5-b36b-a8a77013e365'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.464632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:00.469062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cfec738cab491ba413e062b5a00813', 'request_id': 'req-9ed53cef-729e-42b5-b36b-a8a77013e365'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.476885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cfec738cab491ba413e062b5a00813', 'request_id': 'req-9ed53cef-729e-42b5-b36b-a8a77013e365'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.488266 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] The token cache is being invalidate because project 80cfec738cab491ba413e062b5a00813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:00.488557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.489181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9ed53cef-729e-42b5-b36b-a8a77013e365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:00.490626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1291/2572] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:00 2022] DELETE /identity/v3/projects/80cfec738cab491ba413e062b5a00813 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:00.495563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820b9d4-9b0c-4150-94b8-e88abfdc8bf1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.495880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820b9d4-9b0c-4150-94b8-e88abfdc8bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.496156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820b9d4-9b0c-4150-94b8-e88abfdc8bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.515645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3820b9d4-9b0c-4150-94b8-e88abfdc8bf1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:00.516929 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3820b9d4-9b0c-4150-94b8-e88abfdc8bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:00.578199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1282/2573] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:00.586918 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376c0d1-40fc-49ae-bb04-afad83693843 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.587204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376c0d1-40fc-49ae-bb04-afad83693843 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.587477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376c0d1-40fc-49ae-bb04-afad83693843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.607428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3376c0d1-40fc-49ae-bb04-afad83693843 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:00.608593 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3376c0d1-40fc-49ae-bb04-afad83693843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:00.617460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1292/2574] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:00.621695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820b9d4-9b0c-4150-94b8-e88abfdc8bf1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.631183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820b9d4-9b0c-4150-94b8-e88abfdc8bf1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.635604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.636307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.636582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.636851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.637488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:00.637806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.639373 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.650336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1283/2575] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:00.654321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376c0d1-40fc-49ae-bb04-afad83693843 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.666350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376c0d1-40fc-49ae-bb04-afad83693843 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.671987 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.672574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.672848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.673222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.673856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.676349 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.703620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1293/2576] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:00 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:00.707962 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.718178 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e9ce-e9f5-4383-9ef0-7d2bb60c30c8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.774169 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.774773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.775049 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.775317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.775982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.777864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.786471 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:00.801352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1284/2577] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:00 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:00.805741 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.816285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5780-4367-4e7b-b9e5-fc39c501aeb3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.821127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.821673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.821951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.822221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.823462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.824514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.836262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1294/2578] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:00.840372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.850847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8dfb-4c8f-466e-abad-5a420617d419 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.855425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5def03d-04d1-4251-b423-91839a62b1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.856045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def03d-04d1-4251-b423-91839a62b1ec admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.856329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def03d-04d1-4251-b423-91839a62b1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.856600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def03d-04d1-4251-b423-91839a62b1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.857235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5def03d-04d1-4251-b423-91839a62b1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.858863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5def03d-04d1-4251-b423-91839a62b1ec admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.869054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1285/2579] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:00.872988 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:00.882853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6fe38-41a6-4b6f-bf7a-def7531a6bbe admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:00.887351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:00.888019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.888297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.888569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] PATH_INFO: `/v3/projects/d9b76707e95341e1afe39782eaf02345/users/9a4f547618f2452a896e6004f479575b/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.903024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b76707e95341e1afe39782eaf02345, user_id=9a4f547618f2452a896e6004f479575b, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:00.905651 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:00.919312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1295/2580] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:00 2022] PUT /identity/v3/projects/d9b76707e95341e1afe39782eaf02345/users/9a4f547618f2452a896e6004f479575b/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:00.929821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7702a3-62e3-4955-8b14-fd0534788325 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:00.930211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7702a3-62e3-4955-8b14-fd0534788325 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:00.930622 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7702a3-62e3-4955-8b14-fd0534788325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:00.959058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9a7702a3-62e3-4955-8b14-fd0534788325 None None] MFA Rules not processed for user `9a4f547618f2452a896e6004f479575b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:00.960289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9a7702a3-62e3-4955-8b14-fd0534788325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:01.017515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.020013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:01.029997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1286/2581] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:01.035075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7702a3-62e3-4955-8b14-fd0534788325 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.078374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3569ab2-18ce-4245-bc56-8b94393ac0d1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.082950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.083553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.083861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.084132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.085852 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:01.098388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7702a3-62e3-4955-8b14-fd0534788325 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.102910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.103506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.103814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.104092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.104706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.106588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.133337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1287/2582] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:01 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:01.137354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.152421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.153845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.163466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1296/2583] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:01.189615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:01.201510 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36737e-bb86-49e0-9384-4e27ab1647c9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.207042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.207651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.207969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.208244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.208872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.210860 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.231698 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1288/2584] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:01 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:01.235903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.245624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f986b2-96a1-44b0-875e-c23415e1d253 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.250921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.251677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.251992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.252261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.253484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.255053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.267054 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1297/2585] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:01.272068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.290731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797c8a7-c67b-4dd8-b501-5d274e858aff admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.298634 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.299778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.300339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.300790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] PATH_INFO: `/v3/projects/6a5139098e614f99a244ad1b6541200c/users/8f8d1da54f6a4f30bcc7d9c8c95b3536/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.328936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5139098e614f99a244ad1b6541200c, user_id=8f8d1da54f6a4f30bcc7d9c8c95b3536, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.332647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.348363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1289/2586] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:01 2022] PUT /identity/v3/projects/6a5139098e614f99a244ad1b6541200c/users/8f8d1da54f6a4f30bcc7d9c8c95b3536/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:01.356936 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.369346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11310c3-2197-4857-b5d9-98d273eee3bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.383657 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:01.440939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a3de6-1492-443b-9d5f-66f35c4613d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.441670 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a3de6-1492-443b-9d5f-66f35c4613d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.441989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a3de6-1492-443b-9d5f-66f35c4613d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.442350 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a3de6-1492-443b-9d5f-66f35c4613d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.443062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a3de6-1492-443b-9d5f-66f35c4613d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.444790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a3de6-1492-443b-9d5f-66f35c4613d4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.456342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1298/2587] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:01.460709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.473060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a03a3-cd99-4bd1-b5ec-17deaa33f29b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.478449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.479124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.479404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.479677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] PATH_INFO: `/v3/projects/6a5139098e614f99a244ad1b6541200c/users/8f8d1da54f6a4f30bcc7d9c8c95b3536/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.481829 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5139098e614f99a244ad1b6541200c, user_id=8f8d1da54f6a4f30bcc7d9c8c95b3536, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.484767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.497712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1290/2588] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:01 2022] PUT /identity/v3/projects/6a5139098e614f99a244ad1b6541200c/users/8f8d1da54f6a4f30bcc7d9c8c95b3536/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:01.508684 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac1344-a570-43e0-bd34-7b3feb0d9fef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.509011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac1344-a570-43e0-bd34-7b3feb0d9fef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.509331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac1344-a570-43e0-bd34-7b3feb0d9fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.548510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d6ac1344-a570-43e0-bd34-7b3feb0d9fef None None] MFA Rules not processed for user `8f8d1da54f6a4f30bcc7d9c8c95b3536`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:01.549782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d6ac1344-a570-43e0-bd34-7b3feb0d9fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:01.628550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1299/2589] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:01.640395 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:01.642232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:01.701939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb1a1-4015-4fb9-91dc-39b3ee68c829 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:01.706173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce1add-efee-4eac-b8b3-917b64ba382b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:01.706780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce1add-efee-4eac-b8b3-917b64ba382b service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:01.707057 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce1add-efee-4eac-b8b3-917b64ba382b service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:01.707330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce1add-efee-4eac-b8b3-917b64ba382b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:01.716351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce1add-efee-4eac-b8b3-917b64ba382b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:01.717779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce1add-efee-4eac-b8b3-917b64ba382b service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:01.727988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1291/2590] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:02.151184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac1344-a570-43e0-bd34-7b3feb0d9fef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:02.214839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac1344-a570-43e0-bd34-7b3feb0d9fef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:02.219467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:02.220160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:02.220439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:02.220713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] PATH_INFO: `/v3/users/9a4f547618f2452a896e6004f479575b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:02.221909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4f547618f2452a896e6004f479575b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:02.223847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:02.267061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:02.273966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4f547618f2452a896e6004f479575b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.281255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4f547618f2452a896e6004f479575b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.294144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4f547618f2452a896e6004f479575b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.294838 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4f547618f2452a896e6004f479575b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.303238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4f547618f2452a896e6004f479575b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.309551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1300/2591] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:02 2022] DELETE /identity/v3/users/9a4f547618f2452a896e6004f479575b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:02.314007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:02.314462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:02.314653 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:02.334520 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:02.335739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:02.404164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1292/2592] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:02.416329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:02.417875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:02.490023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e582a-754d-4a99-86d1-de168520c9ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:02.494522 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:02.496212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:02.496828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:02.497105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:02.508687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:02.510147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:02.521104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1301/2593] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:02.811216 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:02.813218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:02.832007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fe499-d1e9-4297-9975-a2f0d9f39194 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:02.837129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:02.837910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:02.838289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:02.838691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] PATH_INFO: `/v3/projects/d9b76707e95341e1afe39782eaf02345` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:02.840017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b76707e95341e1afe39782eaf02345)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:02.842029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:02.902454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:02.908327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b76707e95341e1afe39782eaf02345', 'request_id': 'req-8d652b22-95ad-4635-943d-bcbc1aa83cac'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.916155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b76707e95341e1afe39782eaf02345', 'request_id': 'req-8d652b22-95ad-4635-943d-bcbc1aa83cac'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.925183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b76707e95341e1afe39782eaf02345', 'request_id': 'req-8d652b22-95ad-4635-943d-bcbc1aa83cac'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.937945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] The token cache is being invalidate because project d9b76707e95341e1afe39782eaf02345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:02.938426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.939581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:02.941095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1293/2594] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:02 2022] DELETE /identity/v3/projects/d9b76707e95341e1afe39782eaf02345 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:02.944942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:02.958816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7694d-dabc-4f7a-8d0a-19883bc18ece service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:03.031788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:03.032937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:03.033215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:03.033536 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] PATH_INFO: `/v3/users/8f8d1da54f6a4f30bcc7d9c8c95b3536` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:03.035296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8d1da54f6a4f30bcc7d9c8c95b3536)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:03.037277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:03.098002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8d1da54f6a4f30bcc7d9c8c95b3536'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.106689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8d1da54f6a4f30bcc7d9c8c95b3536'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.113152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:03.122048 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8d1da54f6a4f30bcc7d9c8c95b3536'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.123419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8d1da54f6a4f30bcc7d9c8c95b3536'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.138407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cdb2c726-34cc-4bfb-a08b-09162f342fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8d1da54f6a4f30bcc7d9c8c95b3536'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.146440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1302/2595] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:02 2022] DELETE /identity/v3/users/8f8d1da54f6a4f30bcc7d9c8c95b3536 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:03.630440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:03.632184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:03.645689 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d652b22-95ad-4635-943d-bcbc1aa83cac admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:03.733172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:03.733821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:03.734160 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:03.734442 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] PATH_INFO: `/v3/projects/6a5139098e614f99a244ad1b6541200c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:03.735607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5139098e614f99a244ad1b6541200c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:03.738270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:03.794115 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5139098e614f99a244ad1b6541200c', 'request_id': 'req-4a29aed9-e484-4b9b-85c5-23179beeb52a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.801042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5139098e614f99a244ad1b6541200c', 'request_id': 'req-4a29aed9-e484-4b9b-85c5-23179beeb52a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.811172 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5139098e614f99a244ad1b6541200c', 'request_id': 'req-4a29aed9-e484-4b9b-85c5-23179beeb52a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.823786 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] The token cache is being invalidate because project 6a5139098e614f99a244ad1b6541200c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:03.824398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.824695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4a29aed9-e484-4b9b-85c5-23179beeb52a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:03.826383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1294/2596] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:03 2022] DELETE /identity/v3/projects/6a5139098e614f99a244ad1b6541200c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:03.831033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02811cf7-bda8-488e-a0f0-d630b7e7f2ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:03.831519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02811cf7-bda8-488e-a0f0-d630b7e7f2ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:03.831722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02811cf7-bda8-488e-a0f0-d630b7e7f2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:03.851216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-02811cf7-bda8-488e-a0f0-d630b7e7f2ed None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:03.852476 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02811cf7-bda8-488e-a0f0-d630b7e7f2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:03.914779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1303/2597] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:03.925620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6411e8b-9ec2-402e-9fbf-34d24c7ca142 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:03.926029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6411e8b-9ec2-402e-9fbf-34d24c7ca142 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:03.926366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6411e8b-9ec2-402e-9fbf-34d24c7ca142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:03.947932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b6411e8b-9ec2-402e-9fbf-34d24c7ca142 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:03.949154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b6411e8b-9ec2-402e-9fbf-34d24c7ca142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:03.957279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1295/2598] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:03.961456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02811cf7-bda8-488e-a0f0-d630b7e7f2ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:03.972128 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02811cf7-bda8-488e-a0f0-d630b7e7f2ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:03.977285 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:03.977839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:03.978192 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:03.978568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:03.979387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:03.979774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:03.981538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:03.997662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1304/2599] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:19:04.000683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6411e8b-9ec2-402e-9fbf-34d24c7ca142 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.011064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6411e8b-9ec2-402e-9fbf-34d24c7ca142 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.016345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.017177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.017561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.018091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.018854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.020985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.054881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1296/2600] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:03 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:04.058799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.069447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490a0a3-f8d2-459e-a9d8-4b7f7afaf9fe admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.146343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.146968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.147254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.147525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.153207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.155309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.192592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1305/2601] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:04 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:04.196552 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.209798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bd2d6-a64d-496c-b779-62a37754f15e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.215890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.216439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.216965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.216965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.217612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.219232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.234891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1297/2602] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:04.238628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.253497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb05c7a-754c-4bb3-9384-552257011e69 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.269524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00df78-d20c-458a-b749-e2d84e4672ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.270023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00df78-d20c-458a-b749-e2d84e4672ff admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.270396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00df78-d20c-458a-b749-e2d84e4672ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.270785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00df78-d20c-458a-b749-e2d84e4672ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.271527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00df78-d20c-458a-b749-e2d84e4672ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.273280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00df78-d20c-458a-b749-e2d84e4672ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.289298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1306/2603] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:04.293850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.306255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8747e7-a4ee-461b-a8a4-c77687df8a36 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.313525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.314195 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.314452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.314726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] PATH_INFO: `/v3/projects/aed9fe6e6e6042f0a543b90b4fe2aa0a/users/28cb168e34a64d329c63f2309adf1341/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.331974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed9fe6e6e6042f0a543b90b4fe2aa0a, user_id=28cb168e34a64d329c63f2309adf1341, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.334683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.345891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:04.360193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1298/2604] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:04 2022] PUT /identity/v3/projects/aed9fe6e6e6042f0a543b90b4fe2aa0a/users/28cb168e34a64d329c63f2309adf1341/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:04.366531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.366821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.367104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.404251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] MFA Rules not processed for user `28cb168e34a64d329c63f2309adf1341`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:04.406633 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:04.463191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:04.502903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1307/2605] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:04.507417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.580357 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f683ec6-0a68-4868-9524-0991b2baad85 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.592267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.592267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.592505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.592683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.593330 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.593816 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.595253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.596870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:04.620888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c1cb1-da66-4373-a441-1564311dad79 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.626158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.626831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.627159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.627515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] PATH_INFO: `/v3/users/2dc5c46544a447159b7fd3d225cd9697` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.632822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1299/2606] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:04 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:04.633334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc5c46544a447159b7fd3d225cd9697)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.634127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.634604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.687185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc5c46544a447159b7fd3d225cd9697'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:04.697100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc5c46544a447159b7fd3d225cd9697'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:04.707078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89bc48-c06f-45de-a6d9-c140fd41d0a6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.711599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc5c46544a447159b7fd3d225cd9697'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:04.712950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc5c46544a447159b7fd3d225cd9697'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:04.713281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.714152 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.714432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.714706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.715704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.719045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.721067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc5c46544a447159b7fd3d225cd9697'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:04.737093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1308/2607] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:04 2022] DELETE /identity/v3/users/2dc5c46544a447159b7fd3d225cd9697 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:04.749921 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1300/2608] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:04 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:04.753977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.755606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:04.830081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56184c7-aed4-4e06-999f-78cee9366af3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.835796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.836666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.837164 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.837542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.838338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.840105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.852023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1309/2609] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:04.856398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.867351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd45fb-daec-4bde-a0a4-641dc185b35c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.872086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.872736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.873017 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.873289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] PATH_INFO: `/v3/projects/91b63e48086445e2a1dac70d1d9d2b33/users/57d7ea6b0e1a41288460b9159f03016a/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.889518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b63e48086445e2a1dac70d1d9d2b33, user_id=57d7ea6b0e1a41288460b9159f03016a, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.892280 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.907441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1301/2610] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:04 2022] PUT /identity/v3/projects/91b63e48086445e2a1dac70d1d9d2b33/users/57d7ea6b0e1a41288460b9159f03016a/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:04.911518 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.923466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:04.925420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39892b-7edb-4f37-8fe0-77c826e3ae9b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:04.925568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:04.970876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:04.980054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:04.989247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:04.989862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:04.990146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:04.990422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:04.991063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:04.992695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:04.997285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caff0d-77fc-4a84-976b-a2642b286dc6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.002219 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.003160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1310/2611] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:05.003453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.003844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.004124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] PATH_INFO: `/v3/projects/1c096a50088a4b8dbd00458b324f301e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.005321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c096a50088a4b8dbd00458b324f301e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.007019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.007454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.016715 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c75db-1179-4564-b3ef-a9c86855957f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.022295 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b200e-b12e-4710-b8b5-ad30139a5cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.022975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b200e-b12e-4710-b8b5-ad30139a5cce admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.023284 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b200e-b12e-4710-b8b5-ad30139a5cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.023559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b200e-b12e-4710-b8b5-ad30139a5cce admin admin] PATH_INFO: `/v3/projects/91b63e48086445e2a1dac70d1d9d2b33/users/57d7ea6b0e1a41288460b9159f03016a/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.025499 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b200e-b12e-4710-b8b5-ad30139a5cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b63e48086445e2a1dac70d1d9d2b33, user_id=57d7ea6b0e1a41288460b9159f03016a, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.028153 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b200e-b12e-4710-b8b5-ad30139a5cce admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.041901 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1311/2612] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:05 2022] PUT /identity/v3/projects/91b63e48086445e2a1dac70d1d9d2b33/users/57d7ea6b0e1a41288460b9159f03016a/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:05.052767 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.053055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.053722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.061008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c096a50088a4b8dbd00458b324f301e', 'request_id': 'req-d98eca4d-ce0d-4e92-b988-b5e83611406f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.067416 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c096a50088a4b8dbd00458b324f301e', 'request_id': 'req-d98eca4d-ce0d-4e92-b988-b5e83611406f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.075621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c096a50088a4b8dbd00458b324f301e', 'request_id': 'req-d98eca4d-ce0d-4e92-b988-b5e83611406f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.082445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] MFA Rules not processed for user `57d7ea6b0e1a41288460b9159f03016a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:05.083951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.087541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] The token cache is being invalidate because project 1c096a50088a4b8dbd00458b324f301e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:05.087864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.088501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d98eca4d-ce0d-4e92-b988-b5e83611406f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.089995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1302/2613] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:04 2022] DELETE /identity/v3/projects/1c096a50088a4b8dbd00458b324f301e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:05.094770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358165f-0424-4699-82d6-ba08b975f03d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.095053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358165f-0424-4699-82d6-ba08b975f03d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.095321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358165f-0424-4699-82d6-ba08b975f03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.117186 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9358165f-0424-4699-82d6-ba08b975f03d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:05.118700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9358165f-0424-4699-82d6-ba08b975f03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.156392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1312/2614] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:05.168093 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.169555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.182075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1303/2615] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:05.189702 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b550dc-a228-4e63-bf07-4a760ecfd45f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.190483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b550dc-a228-4e63-bf07-4a760ecfd45f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.190757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b550dc-a228-4e63-bf07-4a760ecfd45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.210063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a2b550dc-a228-4e63-bf07-4a760ecfd45f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:05.211225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a2b550dc-a228-4e63-bf07-4a760ecfd45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.219119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1304/2616] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:05.223154 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b550dc-a228-4e63-bf07-4a760ecfd45f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.228279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e4f3a-d0be-4104-9a13-7b09ab0ed703 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.231846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b550dc-a228-4e63-bf07-4a760ecfd45f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.231982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.232512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.232792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.233061 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.236437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.237521 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.237806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.238080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.238714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:05.239013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.240620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.242506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.243949 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.262503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1305/2617] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:19:05.264605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1313/2618] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:05.266806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.276398 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1d226-f4cc-4d2c-840e-0aef7e9832cf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.280914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.281493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.281767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.282034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.282646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.287896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.292931 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.294317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.309346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d36f9-0151-4058-9ab8-79ff07bc0070 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.314149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.314763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.315696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.315995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] PATH_INFO: `/v3/users/28cb168e34a64d329c63f2309adf1341` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.316113 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1306/2619] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:05.317182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28cb168e34a64d329c63f2309adf1341)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.319087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.319813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.362939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:05.368134 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28cb168e34a64d329c63f2309adf1341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.369929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:05.376759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28cb168e34a64d329c63f2309adf1341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.389356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c577e7-1715-4641-8f6c-86bd4bcefe28 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.390693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28cb168e34a64d329c63f2309adf1341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.391418 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28cb168e34a64d329c63f2309adf1341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.394628 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.395228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.395506 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.395804 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.396429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.398327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.398864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed69111e-f7a9-4c63-a259-195056dbc9f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28cb168e34a64d329c63f2309adf1341'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:05.404944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1314/2620] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:05 2022] DELETE /identity/v3/users/28cb168e34a64d329c63f2309adf1341 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:05.409469 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.409943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.410194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.422006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1307/2621] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:05.422975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.425206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.430600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:05.431788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.436847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898cc8f-d357-4fee-95fa-b22eb05da610 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.491713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1315/2622] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:05.496704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.497319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.497598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.497863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.498498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.500223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.501491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.503480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.511077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1308/2623] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:05.514943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.524476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d26d5-e9ea-4e77-bcfe-cfda14023366 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.529044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.529620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.529900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.530167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.530784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.532394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.543162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1309/2624] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:05.547343 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.557192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2e5fc-dd25-45ef-a2a9-88192f7dfc36 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.560628 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60599b5a-724a-46c8-87a0-c2bb479ed556 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.562034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.562656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.562931 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.563203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] PATH_INFO: `/v3/projects/a623befcf5b44fdda6d8d0d344e982ff/users/1dec53a034914e0e86b8531cd2687218/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.564601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bc1af-3ee1-446f-a550-5f5bb34ac69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.565811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc1af-3ee1-446f-a550-5f5bb34ac69a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.566084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc1af-3ee1-446f-a550-5f5bb34ac69a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.566390 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc1af-3ee1-446f-a550-5f5bb34ac69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.575725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bc1af-3ee1-446f-a550-5f5bb34ac69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.577165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bc1af-3ee1-446f-a550-5f5bb34ac69a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.581313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a623befcf5b44fdda6d8d0d344e982ff, user_id=1dec53a034914e0e86b8531cd2687218, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.584936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.588759 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1316/2625] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:05.599045 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1310/2626] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:05 2022] PUT /identity/v3/projects/a623befcf5b44fdda6d8d0d344e982ff/users/1dec53a034914e0e86b8531cd2687218/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:05.609555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950556af-dc8a-427b-9715-e480003dc0b8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.609842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950556af-dc8a-427b-9715-e480003dc0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.610191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950556af-dc8a-427b-9715-e480003dc0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.639370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-950556af-dc8a-427b-9715-e480003dc0b8 None None] MFA Rules not processed for user `1dec53a034914e0e86b8531cd2687218`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:05.640690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-950556af-dc8a-427b-9715-e480003dc0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.722833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:05.729157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1317/2627] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:05.734011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.820585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56b3e-34cc-402d-bf36-0f7df94ae8a8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.827512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.831418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.831418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.831418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.831418 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.833553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.873066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1311/2628] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:05.874347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950556af-dc8a-427b-9715-e480003dc0b8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.886224 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950556af-dc8a-427b-9715-e480003dc0b8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.900785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.903549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:05.925641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:05.954405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:05.955151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:05.955535 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:05.956328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:05.956971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:05.958991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:05.980976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1318/2629] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:05 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:05.985218 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:05.994421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a9306-888f-4e9d-9337-5f70599a6b01 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.995863 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078f286-ceaf-4451-9c6d-624008f6e786 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:05.999312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.000032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.000315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.000415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.000591 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] PATH_INFO: `/v3/projects/aed9fe6e6e6042f0a543b90b4fe2aa0a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.000950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.001232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.001498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.001784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed9fe6e6e6042f0a543b90b4fe2aa0a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.002133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.004263 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.004389 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.016599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1319/2630] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:06.020512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:06.029778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec70d7-a9f8-4671-b892-85a31a63d09f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:06.035385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.036175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.036546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.036912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] PATH_INFO: `/v3/projects/585113da758f4cd8ba6f73925e8a7a2f/users/ae0379b732884bc394e7456d534df5f8/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.052663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=585113da758f4cd8ba6f73925e8a7a2f, user_id=ae0379b732884bc394e7456d534df5f8, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.055307 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.056904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed9fe6e6e6042f0a543b90b4fe2aa0a', 'request_id': 'req-f4cc985e-63b9-4828-91a5-505907db4821'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.062713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed9fe6e6e6042f0a543b90b4fe2aa0a', 'request_id': 'req-f4cc985e-63b9-4828-91a5-505907db4821'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.069226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1320/2631] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:06 2022] PUT /identity/v3/projects/585113da758f4cd8ba6f73925e8a7a2f/users/ae0379b732884bc394e7456d534df5f8/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:06.071112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed9fe6e6e6042f0a543b90b4fe2aa0a', 'request_id': 'req-f4cc985e-63b9-4828-91a5-505907db4821'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.073059 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:06.082572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] The token cache is being invalidate because project aed9fe6e6e6042f0a543b90b4fe2aa0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:06.082861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.083503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.084993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1312/2632] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:05 2022] DELETE /identity/v3/projects/aed9fe6e6e6042f0a543b90b4fe2aa0a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:06.088743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:06.090139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.140064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39671498-b43d-422c-9cd3-21fc0887f30c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:06.145176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.145787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.146070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.146342 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.146979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.148631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.157703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc985e-63b9-4828-91a5-505907db4821 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:06.158496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1321/2633] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:06.162569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:06.164313 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.165974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.166612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.166893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.167164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] PATH_INFO: `/v3/users/57d7ea6b0e1a41288460b9159f03016a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.168321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d7ea6b0e1a41288460b9159f03016a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.170314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.177441 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4560b-4e5e-4587-b090-3b6b57a98ba1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:06.182228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa2f79-ea0a-4415-8855-9b1d7909921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.182881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2f79-ea0a-4415-8855-9b1d7909921c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.183158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2f79-ea0a-4415-8855-9b1d7909921c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.183434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2f79-ea0a-4415-8855-9b1d7909921c admin admin] PATH_INFO: `/v3/projects/585113da758f4cd8ba6f73925e8a7a2f/users/ae0379b732884bc394e7456d534df5f8/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.185851 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa2f79-ea0a-4415-8855-9b1d7909921c admin admin] RBAC: Authorizing `identity:create_grant(project_id=585113da758f4cd8ba6f73925e8a7a2f, user_id=ae0379b732884bc394e7456d534df5f8, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.188019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa2f79-ea0a-4415-8855-9b1d7909921c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.203216 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1322/2634] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:06 2022] PUT /identity/v3/projects/585113da758f4cd8ba6f73925e8a7a2f/users/ae0379b732884bc394e7456d534df5f8/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:06.214972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.215251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.215521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.226079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:06.227824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d7ea6b0e1a41288460b9159f03016a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.228704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d7ea6b0e1a41288460b9159f03016a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.238364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d7ea6b0e1a41288460b9159f03016a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.243810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] MFA Rules not processed for user `ae0379b732884bc394e7456d534df5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:06.245073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.250311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d7ea6b0e1a41288460b9159f03016a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.250998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d7ea6b0e1a41288460b9159f03016a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.263559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1313/2635] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:06 2022] DELETE /identity/v3/users/57d7ea6b0e1a41288460b9159f03016a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:06.271382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:06.321340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1323/2636] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:06.332298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:06.333919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.394487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf710d-8dba-4d64-a61f-9b2a41564599 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:06.398282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.398957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.399240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.399632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.465917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.468104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.481218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1314/2637] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:06.554854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:06.556290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.608262 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:06.617609 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788e2c9-c4b7-45f8-baa4-f2e69112242c service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:06.621424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e00c3b-2fd4-4503-aa29-b7dc6d83ca44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.622015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e00c3b-2fd4-4503-aa29-b7dc6d83ca44 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.622303 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e00c3b-2fd4-4503-aa29-b7dc6d83ca44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.622576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e00c3b-2fd4-4503-aa29-b7dc6d83ca44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.624325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f7e00c3b-2fd4-4503-aa29-b7dc6d83ca44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.645835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cf348-66c0-4cd5-8a04-e49ad57b53dd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:06.684162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e00c3b-2fd4-4503-aa29-b7dc6d83ca44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.685629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e00c3b-2fd4-4503-aa29-b7dc6d83ca44 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.697485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1315/2638] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:06.704165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:06.704808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.705118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.706049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] PATH_INFO: `/v3/projects/91b63e48086445e2a1dac70d1d9d2b33` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.707198 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b63e48086445e2a1dac70d1d9d2b33)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:06.709268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:06.757568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b63e48086445e2a1dac70d1d9d2b33', 'request_id': 'req-77308d8b-3ff9-44b6-b418-958743ee8fd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.763072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b63e48086445e2a1dac70d1d9d2b33', 'request_id': 'req-77308d8b-3ff9-44b6-b418-958743ee8fd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.770561 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b63e48086445e2a1dac70d1d9d2b33', 'request_id': 'req-77308d8b-3ff9-44b6-b418-958743ee8fd2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.777608 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:06.782707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] The token cache is being invalidate because project 91b63e48086445e2a1dac70d1d9d2b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:06.783002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.783694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77308d8b-3ff9-44b6-b418-958743ee8fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:06.785758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1324/2639] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:06 2022] DELETE /identity/v3/projects/91b63e48086445e2a1dac70d1d9d2b33 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:06.790977 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99316540-344e-4334-8f6a-f678647bb29b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.791512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99316540-344e-4334-8f6a-f678647bb29b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.791512 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99316540-344e-4334-8f6a-f678647bb29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.818823 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-99316540-344e-4334-8f6a-f678647bb29b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:06.820185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-99316540-344e-4334-8f6a-f678647bb29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.826383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:06.929198 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1316/2640] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:06.937412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6e3b-bc5a-47c5-beee-8dab08b13491 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:06.937881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6e3b-bc5a-47c5-beee-8dab08b13491 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:06.938011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6e3b-bc5a-47c5-beee-8dab08b13491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:06.966036 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3dda6e3b-bc5a-47c5-beee-8dab08b13491 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:06.968315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3dda6e3b-bc5a-47c5-beee-8dab08b13491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:06.978014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1325/2641] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:06.984584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99316540-344e-4334-8f6a-f678647bb29b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.000477 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99316540-344e-4334-8f6a-f678647bb29b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.008759 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.009693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.010072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.010569 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.011600 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:07.012107 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.014927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.045997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1317/2642] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:07.049820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6e3b-bc5a-47c5-beee-8dab08b13491 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.059283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6e3b-bc5a-47c5-beee-8dab08b13491 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.065509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.066321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.066658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.066983 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.067844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.069892 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.106792 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1326/2643] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:07 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:07.110986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.124693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4b46e-077c-4fd3-896a-3e42822d79d2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.185944 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.186612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.186895 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.187213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.187889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.190312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.213127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1318/2644] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:07 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:07.217713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.233033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85793aa-76dc-4d09-b71f-f77fe1c65e8a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.238032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.238629 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.238906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.239172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.239833 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.241422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.254722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1327/2645] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:07.256656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.271456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f7c5-3afa-432f-812d-13885aa6bcf3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.276674 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cec4e9-2acc-41ed-ae1d-256639c8d88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.278281 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cec4e9-2acc-41ed-ae1d-256639c8d88d admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.278635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cec4e9-2acc-41ed-ae1d-256639c8d88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.279014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cec4e9-2acc-41ed-ae1d-256639c8d88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.279671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cec4e9-2acc-41ed-ae1d-256639c8d88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.281425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cec4e9-2acc-41ed-ae1d-256639c8d88d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.299387 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1319/2646] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:07.300516 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.314563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564981b8-18d9-4fa6-82fd-a4bcfb0b9ab1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.320415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.321078 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.321360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.321636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] PATH_INFO: `/v3/projects/d45d895be9694a86bdef760fb2a353a0/users/e40f25778fff4f01a4bac58ef34dd563/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.337622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45d895be9694a86bdef760fb2a353a0, user_id=e40f25778fff4f01a4bac58ef34dd563, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.340348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.353828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1328/2647] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:07 2022] PUT /identity/v3/projects/d45d895be9694a86bdef760fb2a353a0/users/e40f25778fff4f01a4bac58ef34dd563/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:07.365206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee32b3-cec1-4544-9b3f-bd679a3c75c3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.365491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee32b3-cec1-4544-9b3f-bd679a3c75c3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.365763 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee32b3-cec1-4544-9b3f-bd679a3c75c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.396657 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8aee32b3-cec1-4544-9b3f-bd679a3c75c3 None None] MFA Rules not processed for user `e40f25778fff4f01a4bac58ef34dd563`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:07.397876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8aee32b3-cec1-4544-9b3f-bd679a3c75c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:07.460695 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1320/2648] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:07.472454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.474207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:07.533303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c9769-c10e-45c3-8422-989ef7e193aa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.536919 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0360c-a6c2-430f-89ea-7a3a3663ddaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.537520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0360c-a6c2-430f-89ea-7a3a3663ddaf service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.537817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0360c-a6c2-430f-89ea-7a3a3663ddaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.538068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0360c-a6c2-430f-89ea-7a3a3663ddaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.546392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0360c-a6c2-430f-89ea-7a3a3663ddaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.547839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0360c-a6c2-430f-89ea-7a3a3663ddaf service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.557055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1329/2649] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:07.576524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee32b3-cec1-4544-9b3f-bd679a3c75c3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.589213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:07.644353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee32b3-cec1-4544-9b3f-bd679a3c75c3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.648825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.649446 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.649723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.649993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] PATH_INFO: `/v3/users/e40f25778fff4f01a4bac58ef34dd563` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.651096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40f25778fff4f01a4bac58ef34dd563)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.653034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.697880 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40f25778fff4f01a4bac58ef34dd563'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:07.698522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40f25778fff4f01a4bac58ef34dd563'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:07.705260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40f25778fff4f01a4bac58ef34dd563'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:07.717425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40f25778fff4f01a4bac58ef34dd563'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:07.718069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40f25778fff4f01a4bac58ef34dd563'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:07.729697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1321/2650] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:07 2022] DELETE /identity/v3/users/e40f25778fff4f01a4bac58ef34dd563 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:07.734174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.734461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.734765 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.755040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:07.756281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:07.813448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1330/2651] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:07.824647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:07.826369 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:07.885407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935e6a1-e568-41ed-b4b9-1976b19aae02 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:07.889123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457530-70c5-4fdb-a586-30745e48d447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:07.889726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457530-70c5-4fdb-a586-30745e48d447 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:07.890007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457530-70c5-4fdb-a586-30745e48d447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:07.890282 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457530-70c5-4fdb-a586-30745e48d447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:07.899203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457530-70c5-4fdb-a586-30745e48d447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:07.900713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457530-70c5-4fdb-a586-30745e48d447 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:07.910602 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1322/2652] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:08.178145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.179971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:08.193165 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb045f-40e4-48b5-bae2-e98e2bbdf9b3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.198111 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.198817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.199428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.199818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] PATH_INFO: `/v3/projects/d45d895be9694a86bdef760fb2a353a0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.201750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45d895be9694a86bdef760fb2a353a0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.204073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.245225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:08.260594 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45d895be9694a86bdef760fb2a353a0', 'request_id': 'req-f0646aa7-0bad-4477-93c9-67e92eec5c7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:08.266562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45d895be9694a86bdef760fb2a353a0', 'request_id': 'req-f0646aa7-0bad-4477-93c9-67e92eec5c7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:08.274581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45d895be9694a86bdef760fb2a353a0', 'request_id': 'req-f0646aa7-0bad-4477-93c9-67e92eec5c7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:08.286458 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] The token cache is being invalidate because project d45d895be9694a86bdef760fb2a353a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:08.286800 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:08.287477 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0646aa7-0bad-4477-93c9-67e92eec5c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:08.289063 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1331/2653] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:08 2022] DELETE /identity/v3/projects/d45d895be9694a86bdef760fb2a353a0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:08.295431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbceabd-aeaa-43b0-892b-e29b7cde36c5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.295720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbceabd-aeaa-43b0-892b-e29b7cde36c5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.296097 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbceabd-aeaa-43b0-892b-e29b7cde36c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.317402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3cbceabd-aeaa-43b0-892b-e29b7cde36c5 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:08.318559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3cbceabd-aeaa-43b0-892b-e29b7cde36c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:08.344525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:08.381450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1323/2654] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:08.390736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843ea17-ea19-4057-80b9-5b90bfc8431b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.391021 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843ea17-ea19-4057-80b9-5b90bfc8431b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.391289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843ea17-ea19-4057-80b9-5b90bfc8431b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.413035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e843ea17-ea19-4057-80b9-5b90bfc8431b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:08.414045 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e843ea17-ea19-4057-80b9-5b90bfc8431b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:08.423038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1332/2655] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:08.427643 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbceabd-aeaa-43b0-892b-e29b7cde36c5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.440349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbceabd-aeaa-43b0-892b-e29b7cde36c5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.445982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.446598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.446882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.447157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.447836 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:08.448143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.449732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.469651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1324/2656] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:08.473726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843ea17-ea19-4057-80b9-5b90bfc8431b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.489846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843ea17-ea19-4057-80b9-5b90bfc8431b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.496112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.496935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.497280 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.497658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.498642 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.500716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.532757 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1333/2657] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:08 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:08.533919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.544093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6b239-683e-4365-81bb-1cab4373d582 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.607407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.608182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.608500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.608801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.609441 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.611451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.639401 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1325/2658] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:08 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:08.643501 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.654173 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b719cf-b53c-4ea6-9b70-0a1eb0c3c5fe admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.659781 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.660407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.660761 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.661103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.661965 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.663597 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.675146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1334/2659] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:08.679220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.689484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70afc-1f85-4dba-8804-d88ea9274b5a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.696092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfe8fa-8aaa-4526-a4f2-ac0cf9505139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.696709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfe8fa-8aaa-4526-a4f2-ac0cf9505139 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.696990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfe8fa-8aaa-4526-a4f2-ac0cf9505139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.697404 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfe8fa-8aaa-4526-a4f2-ac0cf9505139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.698063 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfe8fa-8aaa-4526-a4f2-ac0cf9505139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.699669 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfe8fa-8aaa-4526-a4f2-ac0cf9505139 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.711089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1326/2660] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:08.715438 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.727963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df7ce-ce45-4b91-b961-a95fc0e334a1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.732546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.733189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.733465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.733737 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] PATH_INFO: `/v3/projects/1dcf2fd5ba1e4628b7c77c5ac5f28a54/users/8c20872adb144cdcab29f6b925a52a8f/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.748462 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dcf2fd5ba1e4628b7c77c5ac5f28a54, user_id=8c20872adb144cdcab29f6b925a52a8f, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.751072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.764064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1335/2661] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:08 2022] PUT /identity/v3/projects/1dcf2fd5ba1e4628b7c77c5ac5f28a54/users/8c20872adb144cdcab29f6b925a52a8f/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:08.774011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.774309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.774608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.802755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] MFA Rules not processed for user `8c20872adb144cdcab29f6b925a52a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:08.803987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:08.866505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1327/2662] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:08.877685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:08.879382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:08.936371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982213b6-93be-408e-a70f-2adaf2893b67 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:08.940019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be4af5-599f-4ae5-bef7-5e49863b80ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:08.940624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be4af5-599f-4ae5-bef7-5e49863b80ba service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:08.940905 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be4af5-599f-4ae5-bef7-5e49863b80ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:08.941179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be4af5-599f-4ae5-bef7-5e49863b80ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:08.949223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be4af5-599f-4ae5-bef7-5e49863b80ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:08.950626 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be4af5-599f-4ae5-bef7-5e49863b80ba service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:08.959174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1336/2663] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:09.351380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:09.352841 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:09.380934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:09.417899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d1ce8-2d21-4b0a-9085-a65c8dd18d0c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:09.422312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:09.422930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:09.423208 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:09.423478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] PATH_INFO: `/v3/users/1dec53a034914e0e86b8531cd2687218` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:09.424658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dec53a034914e0e86b8531cd2687218)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:09.426554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:09.470728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dec53a034914e0e86b8531cd2687218'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.471414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dec53a034914e0e86b8531cd2687218'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.478069 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dec53a034914e0e86b8531cd2687218'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.490321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dec53a034914e0e86b8531cd2687218'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.490978 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dec53a034914e0e86b8531cd2687218'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.503694 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1328/2664] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:09 2022] DELETE /identity/v3/users/1dec53a034914e0e86b8531cd2687218 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:09.508202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:09.508678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:09.508787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:09.528056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:09.529281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:09.556782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:09.558206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:09.606610 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:09.617434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1337/2665] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:09.626971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:09.628598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a0c4-b516-4fbc-9de8-1f51d851644b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:09.629018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:09.633246 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:09.633882 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:09.634164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:09.634437 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] PATH_INFO: `/v3/users/2bd1f98efdf04a7192be4211b68d6ff6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:09.635568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd1f98efdf04a7192be4211b68d6ff6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:09.637535 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:09.685690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:09.687632 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd1f98efdf04a7192be4211b68d6ff6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.688554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd1f98efdf04a7192be4211b68d6ff6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.688735 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fb69c-323b-4c5d-b8b6-58d19c462abc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:09.693332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17445071-a480-41ea-931f-a421f45c1ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:09.693950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445071-a480-41ea-931f-a421f45c1ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:09.694263 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445071-a480-41ea-931f-a421f45c1ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:09.694591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445071-a480-41ea-931f-a421f45c1ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:09.696552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-17445071-a480-41ea-931f-a421f45c1ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:09.698264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd1f98efdf04a7192be4211b68d6ff6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.709044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17445071-a480-41ea-931f-a421f45c1ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:09.710869 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17445071-a480-41ea-931f-a421f45c1ae7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:09.711886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd1f98efdf04a7192be4211b68d6ff6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.712573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd1f98efdf04a7192be4211b68d6ff6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:09.721207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1338/2666] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:09.725429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1329/2667] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:09 2022] DELETE /identity/v3/users/2bd1f98efdf04a7192be4211b68d6ff6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:09.729819 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:09.730129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:09.730405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:09.750039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:09.751247 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:09.813275 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1339/2668] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:09.824073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:09.825472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:09.887949 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5d2e-12fb-4bf3-9b0b-b434c849edfa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:09.892189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:09.893423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:09.893792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:09.894075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:09.903436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:09.904904 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:09.915811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1330/2669] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:10.005373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:10.006848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:10.019808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad8b62-f0a4-40a6-b1ce-a170111a5d35 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:10.024979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:10.025618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:10.026243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:10.026243 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] PATH_INFO: `/v3/projects/a623befcf5b44fdda6d8d0d344e982ff` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:10.027305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a623befcf5b44fdda6d8d0d344e982ff)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:10.029310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:10.079023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a623befcf5b44fdda6d8d0d344e982ff', 'request_id': 'req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.085131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a623befcf5b44fdda6d8d0d344e982ff', 'request_id': 'req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.093445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a623befcf5b44fdda6d8d0d344e982ff', 'request_id': 'req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.106131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] The token cache is being invalidate because project a623befcf5b44fdda6d8d0d344e982ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:10.106443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.107074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.108578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1340/2670] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:10 2022] DELETE /identity/v3/projects/a623befcf5b44fdda6d8d0d344e982ff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:10.113325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:10.125636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59dd13-122a-41dc-81d7-765c5898ae7e service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:10.187034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:10.188036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:10.188314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:10.188581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] PATH_INFO: `/v3/users/ae0379b732884bc394e7456d534df5f8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:10.189741 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0379b732884bc394e7456d534df5f8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:10.191676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:10.223294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:10.225265 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:10.241305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d7cd4-2690-4d6f-9f44-a8a6153d00a6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:10.246465 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:10.247222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:10.247596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:10.248399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] PATH_INFO: `/v3/projects/46b57794e0fb4534ac7a082037f79edd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:10.249444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b57794e0fb4534ac7a082037f79edd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:10.251555 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:10.293745 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:10.328577 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b57794e0fb4534ac7a082037f79edd', 'request_id': 'req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.335692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b57794e0fb4534ac7a082037f79edd', 'request_id': 'req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.347602 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b57794e0fb4534ac7a082037f79edd', 'request_id': 'req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.364961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] The token cache is being invalidate because project 46b57794e0fb4534ac7a082037f79edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:10.365152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.366136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.367844 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1341/2671] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:10 2022] DELETE /identity/v3/projects/46b57794e0fb4534ac7a082037f79edd => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:10.370990 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0379b732884bc394e7456d534df5f8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.371563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:10.372517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0379b732884bc394e7456d534df5f8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.372876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:10.382852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0379b732884bc394e7456d534df5f8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.393212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:10.395301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0379b732884bc394e7456d534df5f8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.396193 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0379b732884bc394e7456d534df5f8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.410862 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1331/2672] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:10 2022] DELETE /identity/v3/users/ae0379b732884bc394e7456d534df5f8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:10.483103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1c77f-84a8-4bb4-9127-f621b1fb7f35 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:10.550763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:10.551397 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:10.551663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:10.551991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] PATH_INFO: `/v3/users/be045ae9af954a05b9d62148f4e6da1d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:10.553244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be045ae9af954a05b9d62148f4e6da1d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:10.555099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:10.611385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be045ae9af954a05b9d62148f4e6da1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.621664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be045ae9af954a05b9d62148f4e6da1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.628386 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:10.639646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be045ae9af954a05b9d62148f4e6da1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.641506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be045ae9af954a05b9d62148f4e6da1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.652870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7c5227c-aa60-4f29-a1a2-7153d63a5727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be045ae9af954a05b9d62148f4e6da1d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.659802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1342/2673] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:10 2022] DELETE /identity/v3/users/be045ae9af954a05b9d62148f4e6da1d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:10.757983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:10.759445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:10.773090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49075d60-47e8-4059-988c-5fdd3080b47b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:10.860149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:10.861040 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:10.861386 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:10.862413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] PATH_INFO: `/v3/projects/585113da758f4cd8ba6f73925e8a7a2f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:10.868976 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=585113da758f4cd8ba6f73925e8a7a2f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:10.871790 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:10.956213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585113da758f4cd8ba6f73925e8a7a2f', 'request_id': 'req-593c279e-093a-48f5-af19-2456b4240856'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.964926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585113da758f4cd8ba6f73925e8a7a2f', 'request_id': 'req-593c279e-093a-48f5-af19-2456b4240856'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.974671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585113da758f4cd8ba6f73925e8a7a2f', 'request_id': 'req-593c279e-093a-48f5-af19-2456b4240856'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.987495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] The token cache is being invalidate because project 585113da758f4cd8ba6f73925e8a7a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:10.987780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.988490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:10.990036 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1332/2674] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:10 2022] DELETE /identity/v3/projects/585113da758f4cd8ba6f73925e8a7a2f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:10.995261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a70a9c-4931-484e-b12f-088a30884af1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:10.995551 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a70a9c-4931-484e-b12f-088a30884af1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:10.995861 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a70a9c-4931-484e-b12f-088a30884af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.013639 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.015668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.018346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e7a70a9c-4931-484e-b12f-088a30884af1 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:11.019578 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e7a70a9c-4931-484e-b12f-088a30884af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.085713 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c279e-093a-48f5-af19-2456b4240856 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.089674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1343/2675] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:11.090540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.091168 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.091443 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.091709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] PATH_INFO: `/v3/projects/8b063a33352e41959909c1604472f0c8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.092857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b063a33352e41959909c1604472f0c8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.094736 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.098385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa4d06-c7f6-4806-bcc4-6ca9163188bc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.098680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa4d06-c7f6-4806-bcc4-6ca9163188bc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.098957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa4d06-c7f6-4806-bcc4-6ca9163188bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.112558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:11.121297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-32aa4d06-c7f6-4806-bcc4-6ca9163188bc None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:11.122502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-32aa4d06-c7f6-4806-bcc4-6ca9163188bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.131102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1344/2676] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:11.134979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa4d06-c7f6-4806-bcc4-6ca9163188bc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.143650 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa4d06-c7f6-4806-bcc4-6ca9163188bc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.147347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b063a33352e41959909c1604472f0c8', 'request_id': 'req-a17b1ad8-a055-4f2b-9fef-ee799d326a53'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:11.154321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b063a33352e41959909c1604472f0c8', 'request_id': 'req-a17b1ad8-a055-4f2b-9fef-ee799d326a53'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:11.163092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b063a33352e41959909c1604472f0c8', 'request_id': 'req-a17b1ad8-a055-4f2b-9fef-ee799d326a53'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:11.176319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] The token cache is being invalidate because project 8b063a33352e41959909c1604472f0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:11.176612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:11.177273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a17b1ad8-a055-4f2b-9fef-ee799d326a53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:11.178995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1333/2677] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:11 2022] DELETE /identity/v3/projects/8b063a33352e41959909c1604472f0c8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:11.184315 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cb80b-8f9b-4d44-820c-3b478a40091c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.184601 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cb80b-8f9b-4d44-820c-3b478a40091c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.184922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cb80b-8f9b-4d44-820c-3b478a40091c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.203515 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.204158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.204287 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ed5cb80b-8f9b-4d44-820c-3b478a40091c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:11.204440 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.204712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.205354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:11.205479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ed5cb80b-8f9b-4d44-820c-3b478a40091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.205677 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.207220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.213184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1334/2678] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:11.222011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee95d-0d9d-4fd7-9884-ee5490466901 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.222450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee95d-0d9d-4fd7-9884-ee5490466901 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.222852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee95d-0d9d-4fd7-9884-ee5490466901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.227159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1345/2679] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:11.228137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.229736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.242875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57bafb-d991-4df3-ac90-edd2a44a457e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.243264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-714ee95d-0d9d-4fd7-9884-ee5490466901 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:11.244454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-714ee95d-0d9d-4fd7-9884-ee5490466901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.247337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.247976 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.248256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.248525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.249142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.250999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.252425 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1335/2680] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:11.256482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee95d-0d9d-4fd7-9884-ee5490466901 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.265578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee95d-0d9d-4fd7-9884-ee5490466901 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.271078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.271680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.271982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.272255 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.272885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:11.273182 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.274757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.281219 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1346/2681] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:11.282163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.287229 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1336/2682] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:19:11.291785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.301433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5373f46-27a4-4d79-9e2b-9a2ec338aaff admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.326898 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:11.347359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413196-1666-4989-9dd2-5de381db8b50 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.351974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.352564 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.352853 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.353121 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.353739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.355622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.359142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.359744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.360055 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.360326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.360938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.362885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.376125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1347/2683] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:11.380161 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.390129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae167b-9715-464e-93df-6ee80ee583cf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.391873 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1337/2684] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:11.397100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.448056 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.448652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.448932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.449205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.449828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.451396 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.460385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a07aed-50da-4794-9190-f5c16cf15531 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.462176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1348/2685] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:11.465143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.465746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.465859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.466027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.466299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.466926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.468965 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.476171 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6f9df-2ba5-4bf9-a7f6-ea50a43466cf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.479522 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:11.482299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.482953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.483279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.483603 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.484267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.485950 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.496407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1338/2686] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:11.496557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1349/2687] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:11.498447 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.500432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.508321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cceae73-4aee-4ab8-bd87-1f485b6c8117 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.510049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df71c39-94d0-4c4a-ac9a-ef08e5be42f6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.512941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.513548 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.513784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.514047 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.514668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.514788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29860f3-baba-4d4f-a6dd-55c2ef0789ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.515343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29860f3-baba-4d4f-a6dd-55c2ef0789ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.515622 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29860f3-baba-4d4f-a6dd-55c2ef0789ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.515932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29860f3-baba-4d4f-a6dd-55c2ef0789ba admin admin] PATH_INFO: `/v3/projects/159fa03e2c6c4039a9d38509e576337c/users/f3f304ad9d644ec7a31f0ba1e8fdd2dd/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.516394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.527189 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1339/2688] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:11.531002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.535802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29860f3-baba-4d4f-a6dd-55c2ef0789ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=159fa03e2c6c4039a9d38509e576337c, user_id=f3f304ad9d644ec7a31f0ba1e8fdd2dd, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.538420 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29860f3-baba-4d4f-a6dd-55c2ef0789ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.541950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03949-1c76-4bd3-907b-bbc72575f88f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.547214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.548002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.548279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.548549 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.549275 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.550970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.551554 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1350/2689] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:11 2022] PUT /identity/v3/projects/159fa03e2c6c4039a9d38509e576337c/users/f3f304ad9d644ec7a31f0ba1e8fdd2dd/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:11.564192 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1340/2690] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:11.564381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2b852-aba7-4eea-b887-c2d31bc15b12 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.564381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2b852-aba7-4eea-b887-c2d31bc15b12 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.564381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2b852-aba7-4eea-b887-c2d31bc15b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.565210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.574982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58e0e5-807b-43e1-a611-cabb2927ac88 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.593878 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6bb2b852-aba7-4eea-b887-c2d31bc15b12 None None] MFA Rules not processed for user `f3f304ad9d644ec7a31f0ba1e8fdd2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:11.595090 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6bb2b852-aba7-4eea-b887-c2d31bc15b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.632943 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54eaa5e-9b69-429e-87cc-5d97da4ab18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.633585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54eaa5e-9b69-429e-87cc-5d97da4ab18f admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.633862 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54eaa5e-9b69-429e-87cc-5d97da4ab18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.634132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54eaa5e-9b69-429e-87cc-5d97da4ab18f admin admin] PATH_INFO: `/v3/projects/78c01f014575468d96d56580eb2e07d1/users/42be3298596e4339af40fd38703e25c0/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.648440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54eaa5e-9b69-429e-87cc-5d97da4ab18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c01f014575468d96d56580eb2e07d1, user_id=42be3298596e4339af40fd38703e25c0, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.651062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54eaa5e-9b69-429e-87cc-5d97da4ab18f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.659834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1351/2691] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:11.664298 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2b852-aba7-4eea-b887-c2d31bc15b12 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.664579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1341/2692] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:11 2022] PUT /identity/v3/projects/78c01f014575468d96d56580eb2e07d1/users/42be3298596e4339af40fd38703e25c0/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:11.675560 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51c2f-d809-44ff-98f0-6dc098bf41ab None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.675980 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51c2f-d809-44ff-98f0-6dc098bf41ab None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.676259 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51c2f-d809-44ff-98f0-6dc098bf41ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.678097 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2b852-aba7-4eea-b887-c2d31bc15b12 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.696662 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:11.704519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f1e51c2f-d809-44ff-98f0-6dc098bf41ab None None] MFA Rules not processed for user `42be3298596e4339af40fd38703e25c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:11.706611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f1e51c2f-d809-44ff-98f0-6dc098bf41ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:11.736038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.737182 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.737456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.737725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.738346 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.740270 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.767576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1352/2693] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:11.772039 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.773454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1342/2694] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:11.777554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51c2f-d809-44ff-98f0-6dc098bf41ab None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.787423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51c2f-d809-44ff-98f0-6dc098bf41ab None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.828075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:11.833960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db363c7e-1872-4b26-af70-e3d0482e89fa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.838599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.839424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.839559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.839772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.840409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.842288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.844854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.845456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.845775 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.846048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.846666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.848559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.862888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1353/2695] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:11.866796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.876685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82273233-75ad-464a-bcf3-82fd0016706e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.881472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1343/2696] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:11.882917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.936024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.936636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.937077 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.937204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.938474 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.940579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.953973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1354/2697] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:11.954142 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ee8ae-3c2d-4f6b-93b8-9eedd37b8119 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.956213 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.957371 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.957981 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.958260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.958533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.959162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:11.961101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:11.968929 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640c094-422d-4617-a9a4-504c8d1f29ff admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:11.974468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:11.975310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:11.975588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:11.975895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] PATH_INFO: `/v3/projects/094ae440a2144c33af10e39edf55c1c3/users/1535f870655444a28fe2428125085de1/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:11.990173 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1344/2698] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:11 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:11.991726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:11.997149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094ae440a2144c33af10e39edf55c1c3, user_id=1535f870655444a28fe2428125085de1, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.000264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.001605 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774dcb5-087d-4e28-bb3d-3def182673fa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.007678 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.008372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.008765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.009027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.009798 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.011478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.020364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1355/2699] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:11 2022] PUT /identity/v3/projects/094ae440a2144c33af10e39edf55c1c3/users/1535f870655444a28fe2428125085de1/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:12.023333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1345/2700] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:12.026797 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.028060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.036906 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7205ce-bc23-42d9-bba1-e8b2db64f3ae admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.040004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:12.105482 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.106648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.106995 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.107213 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] PATH_INFO: `/v3/projects/4442ad44d92a477db7472cf0cfe7136b/users/60ac0da791de45d5b23ef3a3f4dd0ce6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.113550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ec522-6927-46f1-813a-d9fde3bc2066 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.121946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.122562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.122912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.123169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.123911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.125533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.125724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4442ad44d92a477db7472cf0cfe7136b, user_id=60ac0da791de45d5b23ef3a3f4dd0ce6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.129077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.143332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1356/2701] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:12.144221 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.146968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1346/2702] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:12 2022] PUT /identity/v3/projects/4442ad44d92a477db7472cf0cfe7136b/users/60ac0da791de45d5b23ef3a3f4dd0ce6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:12.150724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.154288 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7248ed-e15a-46d7-ba52-2cb002b5337a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.195122 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:12.212866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349421a-5842-4e9b-a0df-7102856b6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.213545 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349421a-5842-4e9b-a0df-7102856b6294 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.213824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349421a-5842-4e9b-a0df-7102856b6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.214103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349421a-5842-4e9b-a0df-7102856b6294 admin admin] PATH_INFO: `/v3/projects/094ae440a2144c33af10e39edf55c1c3/users/1535f870655444a28fe2428125085de1/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.216073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349421a-5842-4e9b-a0df-7102856b6294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094ae440a2144c33af10e39edf55c1c3, user_id=1535f870655444a28fe2428125085de1, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.218694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349421a-5842-4e9b-a0df-7102856b6294 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.225648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71495-3c55-411d-b15b-ccfb31f63392 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.229568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1357/2703] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:12 2022] PUT /identity/v3/projects/094ae440a2144c33af10e39edf55c1c3/users/1535f870655444a28fe2428125085de1/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:12.240960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.241820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.241820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.271847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] MFA Rules not processed for user `1535f870655444a28fe2428125085de1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:12.273013 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.295889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.296523 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.296802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.297073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.297710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.299871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.310079 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1347/2704] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:12.314060 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.324800 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a5efc-2a79-4178-9484-24b8fa3ed9ba admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.329987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e762eb-a39b-416d-97d3-7e3917a635fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.330677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e762eb-a39b-416d-97d3-7e3917a635fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.330953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e762eb-a39b-416d-97d3-7e3917a635fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.331224 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e762eb-a39b-416d-97d3-7e3917a635fb admin admin] PATH_INFO: `/v3/projects/4442ad44d92a477db7472cf0cfe7136b/users/60ac0da791de45d5b23ef3a3f4dd0ce6/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.333151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e762eb-a39b-416d-97d3-7e3917a635fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4442ad44d92a477db7472cf0cfe7136b, user_id=60ac0da791de45d5b23ef3a3f4dd0ce6, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.335791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e762eb-a39b-416d-97d3-7e3917a635fb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.346385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1348/2705] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:12 2022] PUT /identity/v3/projects/4442ad44d92a477db7472cf0cfe7136b/users/60ac0da791de45d5b23ef3a3f4dd0ce6/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:12.354190 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1358/2706] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:12.356417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.356725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.357004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.362407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.363906 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.386389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] MFA Rules not processed for user `60ac0da791de45d5b23ef3a3f4dd0ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:12.387625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.392426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:12.423818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022cbef-7249-4405-9de4-0eecad08c35d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.456428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1349/2707] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:12.467611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.469718 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.472888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.473552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.473854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.474159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.482126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e171b-f8dd-4b7c-acf8-d6a3ef98d393 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.485749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.486363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.487162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.487440 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.496429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.497854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.508561 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1350/2708] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:12.535027 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.536527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.547309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1359/2709] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:12.580644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.622351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:12.653121 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6cdae-1ad6-4103-be1b-a3d942e76211 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.657892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.659059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.659363 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.659633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] PATH_INFO: `/v3/users/42be3298596e4339af40fd38703e25c0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.660826 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=42be3298596e4339af40fd38703e25c0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.666801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.690812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.701590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f544d-e43c-41ca-8440-b4e3fd9b5b6e service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.707927 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.708563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.708845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.709118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] PATH_INFO: `/v3/users/f3f304ad9d644ec7a31f0ba1e8fdd2dd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.710274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f304ad9d644ec7a31f0ba1e8fdd2dd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.712624 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.719537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42be3298596e4339af40fd38703e25c0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.720348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42be3298596e4339af40fd38703e25c0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.727734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42be3298596e4339af40fd38703e25c0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.742194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42be3298596e4339af40fd38703e25c0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.742885 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1bb446bf-b537-410a-9d20-e351f3f22acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42be3298596e4339af40fd38703e25c0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.756870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1351/2710] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:12 2022] DELETE /identity/v3/users/42be3298596e4339af40fd38703e25c0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:12.761124 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.761414 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.761686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.762400 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f304ad9d644ec7a31f0ba1e8fdd2dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.768998 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f304ad9d644ec7a31f0ba1e8fdd2dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.780458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:12.781212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f304ad9d644ec7a31f0ba1e8fdd2dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.781594 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.781937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f304ad9d644ec7a31f0ba1e8fdd2dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.788796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ee19a90f-2787-4125-bd2d-88b1b0b38509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f304ad9d644ec7a31f0ba1e8fdd2dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:12.794903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1360/2711] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:12 2022] DELETE /identity/v3/users/f3f304ad9d644ec7a31f0ba1e8fdd2dd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:12.799875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.800169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.800449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.807780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:12.822961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:12.824202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.840564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1352/2712] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:12.850307 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.851693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.884272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1361/2713] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:12.894115 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:12.895787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:12.909183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f23f4-f8ce-4c96-8c2b-002ec7e366fe None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.916994 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:12.960779 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcb685-e46c-4665-ba14-afb0316b1250 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:12.964443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.964951 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:12.965409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.965409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.965579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:12.965675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.965825 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:12.966083 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:12.974856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.975018 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:12.976478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.976645 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:12.986104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1353/2714] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:12.988191 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1362/2715] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:13.266487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:13.268188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:13.275426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:13.277303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:13.281972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b5691-2974-4b8b-9f1e-6870060a3750 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:13.288708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:13.290339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:13.291076 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:13.291514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] PATH_INFO: `/v3/projects/78c01f014575468d96d56580eb2e07d1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:13.293038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78c01f014575468d96d56580eb2e07d1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:13.295411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:13.312424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75293f00-195a-4e6f-b993-5511eaaf0cf8 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:13.334368 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:13.335089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:13.335407 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:13.335739 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] PATH_INFO: `/v3/projects/159fa03e2c6c4039a9d38509e576337c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:13.336891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=159fa03e2c6c4039a9d38509e576337c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:13.339408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:13.358883 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78c01f014575468d96d56580eb2e07d1', 'request_id': 'req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.365540 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78c01f014575468d96d56580eb2e07d1', 'request_id': 'req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.372788 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78c01f014575468d96d56580eb2e07d1', 'request_id': 'req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.384647 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] The token cache is being invalidate because project 78c01f014575468d96d56580eb2e07d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:13.384937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.385677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.387779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1354/2716] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:13 2022] DELETE /identity/v3/projects/78c01f014575468d96d56580eb2e07d1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:13.389194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '159fa03e2c6c4039a9d38509e576337c', 'request_id': 'req-579d549b-fa87-46c9-992e-06ee4685d9e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.391274 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:13.392737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:13.395159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '159fa03e2c6c4039a9d38509e576337c', 'request_id': 'req-579d549b-fa87-46c9-992e-06ee4685d9e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.404525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '159fa03e2c6c4039a9d38509e576337c', 'request_id': 'req-579d549b-fa87-46c9-992e-06ee4685d9e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.419248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] The token cache is being invalidate because project 159fa03e2c6c4039a9d38509e576337c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:13.419529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.420317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.422151 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1363/2717] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:13 2022] DELETE /identity/v3/projects/159fa03e2c6c4039a9d38509e576337c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:13.426687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:13.428067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:13.463680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cf5b6-444e-4ffb-9c7d-0e8c119b0189 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:13.465948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:13.504135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d549b-fa87-46c9-992e-06ee4685d9e2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:13.505445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:13.509992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:13.510658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:13.510962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:13.511308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] PATH_INFO: `/v3/users/1535f870655444a28fe2428125085de1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:13.512618 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1535f870655444a28fe2428125085de1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:13.515339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:13.523352 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:13.524012 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:13.524295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:13.524568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] PATH_INFO: `/v3/users/60ac0da791de45d5b23ef3a3f4dd0ce6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:13.525729 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ac0da791de45d5b23ef3a3f4dd0ce6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:13.527620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:13.565472 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1535f870655444a28fe2428125085de1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.573007 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ac0da791de45d5b23ef3a3f4dd0ce6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.573579 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1535f870655444a28fe2428125085de1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.573722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ac0da791de45d5b23ef3a3f4dd0ce6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.591065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ac0da791de45d5b23ef3a3f4dd0ce6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.591475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1535f870655444a28fe2428125085de1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.592204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1535f870655444a28fe2428125085de1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.599804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1535f870655444a28fe2428125085de1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.605454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ac0da791de45d5b23ef3a3f4dd0ce6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.606218 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ac0da791de45d5b23ef3a3f4dd0ce6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:13.606885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1364/2718] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:13 2022] DELETE /identity/v3/users/1535f870655444a28fe2428125085de1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:13.621533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1355/2719] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:13 2022] DELETE /identity/v3/users/60ac0da791de45d5b23ef3a3f4dd0ce6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:13.902447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:13.905114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:13.910024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:13.911668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:13.982483 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d827544-c96c-4b71-b98e-66ef8121786b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:13.987194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:13.987869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:13.988171 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:13.988466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] PATH_INFO: `/v3/projects/4442ad44d92a477db7472cf0cfe7136b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:13.989579 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4442ad44d92a477db7472cf0cfe7136b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:13.991465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:13.995408 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f819653-e60a-4874-b487-ef4459bb6742 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.000008 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.000647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.000924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.001193 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] PATH_INFO: `/v3/projects/094ae440a2144c33af10e39edf55c1c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.002854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=094ae440a2144c33af10e39edf55c1c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.004772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.045325 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4442ad44d92a477db7472cf0cfe7136b', 'request_id': 'req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.051223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4442ad44d92a477db7472cf0cfe7136b', 'request_id': 'req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.053938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094ae440a2144c33af10e39edf55c1c3', 'request_id': 'req-1a52588f-dfa3-4b43-bce4-f41e924e23e5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.055208 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:14.055573 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:14.061102 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4442ad44d92a477db7472cf0cfe7136b', 'request_id': 'req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.061646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094ae440a2144c33af10e39edf55c1c3', 'request_id': 'req-1a52588f-dfa3-4b43-bce4-f41e924e23e5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.072255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094ae440a2144c33af10e39edf55c1c3', 'request_id': 'req-1a52588f-dfa3-4b43-bce4-f41e924e23e5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.075611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] The token cache is being invalidate because project 4442ad44d92a477db7472cf0cfe7136b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:14.075928 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.076556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7bb175f6-1184-4bf0-b5c6-90b20bc0e254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.080058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1356/2720] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:13 2022] DELETE /identity/v3/projects/4442ad44d92a477db7472cf0cfe7136b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:14.084786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] The token cache is being invalidate because project 094ae440a2144c33af10e39edf55c1c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:14.085079 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.085237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286d37-a4c6-4232-8b27-39f8a9b688fa None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.085412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286d37-a4c6-4232-8b27-39f8a9b688fa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.085684 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286d37-a4c6-4232-8b27-39f8a9b688fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.085842 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1a52588f-dfa3-4b43-bce4-f41e924e23e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:14.087505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1365/2721] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:13 2022] DELETE /identity/v3/projects/094ae440a2144c33af10e39edf55c1c3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:14.091787 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecd476-f142-46cf-b3a1-f3e8e56ac18b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.092085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecd476-f142-46cf-b3a1-f3e8e56ac18b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.092415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecd476-f142-46cf-b3a1-f3e8e56ac18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.105509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ab286d37-a4c6-4232-8b27-39f8a9b688fa None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:14.106732 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ab286d37-a4c6-4232-8b27-39f8a9b688fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:14.112764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-80ecd476-f142-46cf-b3a1-f3e8e56ac18b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:14.114353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80ecd476-f142-46cf-b3a1-f3e8e56ac18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:14.168875 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1357/2722] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:14.178558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a5c40-8d5a-4d76-bdfe-d213566f90cb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.178843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a5c40-8d5a-4d76-bdfe-d213566f90cb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.179119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a5c40-8d5a-4d76-bdfe-d213566f90cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.179675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1366/2723] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:14.187238 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596809-96df-44e7-8ae4-68c8c69e3e7a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.187534 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596809-96df-44e7-8ae4-68c8c69e3e7a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.187824 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e596809-96df-44e7-8ae4-68c8c69e3e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.202261 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d37a5c40-8d5a-4d76-bdfe-d213566f90cb None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:14.203779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d37a5c40-8d5a-4d76-bdfe-d213566f90cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:14.207744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1e596809-96df-44e7-8ae4-68c8c69e3e7a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:14.208925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1e596809-96df-44e7-8ae4-68c8c69e3e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:14.212524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1358/2724] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:14.217257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1367/2725] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:14.218410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a5c40-8d5a-4d76-bdfe-d213566f90cb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.221733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596809-96df-44e7-8ae4-68c8c69e3e7a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.228406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a5c40-8d5a-4d76-bdfe-d213566f90cb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.230847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e596809-96df-44e7-8ae4-68c8c69e3e7a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.233129 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.233770 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.234043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.234359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.234999 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:14.235297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.237050 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.237452 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.237712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.237996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.238268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.239011 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:14.239215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.240826 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.249289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1359/2726] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:14.252755 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1368/2727] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:14.253839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.257810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.264539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cfc86-dade-469e-a068-c3c7533b32aa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.269513 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f4b98-ef92-44c0-8aac-48b2d160b19f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.269975 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.270584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.270864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.271136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.271779 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.273801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.274169 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.274763 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.275042 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.275314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.275960 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.277823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.303048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1360/2728] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:14.308338 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1369/2729] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:14.308686 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.312268 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.379013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105aa2df-2b23-4bd1-b617-9c9d851cbd03 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.391204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20bd42-1e70-4281-a062-de6c1321ebda admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.397125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.397969 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.398348 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.398689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.400932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.402935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.428685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1370/2730] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/users => generated 305 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:14.432823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.440409 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:14.444424 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86124b-d42a-45e3-b041-4badc93d5e14 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.450290 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.450891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.451013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.451172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.451507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.451640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.451816 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.452086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.452212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.452716 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.453859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.454630 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.465357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1371/2731] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:14.469720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.482041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fb35-b4b3-43e0-bd91-dfe6a7aafdb5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.483243 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1361/2732] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:14.488298 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.491868 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.493258 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.493674 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.493991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.495070 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.498211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.506453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef2a47-49b4-48e6-9f62-c31ff5a231bf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.512525 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.513220 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.513614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.513887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.514534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.516536 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.520139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1372/2733] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:14.521246 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.528233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:14.532081 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1362/2734] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:14.532714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c16dc-b006-4530-b5ac-7cc681f4a9bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.536849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.542301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38b6ac-4148-4685-8621-48b294da701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.543245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b6ac-4148-4685-8621-48b294da701d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.543691 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b6ac-4148-4685-8621-48b294da701d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.544232 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b6ac-4148-4685-8621-48b294da701d admin admin] PATH_INFO: `/v3/projects/0ac06f0dea4f4496ba814e81d3b72cb8/users/547083e3258c4745a4c5d08db87942dc/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.550020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e9c39-f51b-4b03-8a83-640e5bb0cc7d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.556728 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.557613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.557613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.557927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.558582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.560233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.563742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38b6ac-4148-4685-8621-48b294da701d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac06f0dea4f4496ba814e81d3b72cb8, user_id=547083e3258c4745a4c5d08db87942dc, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.567505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38b6ac-4148-4685-8621-48b294da701d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.572415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1363/2735] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:14.576554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.589304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ec7b4-1799-407e-a4c2-0563d5b89207 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.609318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdc1af-22a5-4318-868c-83dd58d865a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.609318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdc1af-22a5-4318-868c-83dd58d865a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.609318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdc1af-22a5-4318-868c-83dd58d865a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.609318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdc1af-22a5-4318-868c-83dd58d865a1 admin admin] PATH_INFO: `/v3/projects/e5cce08076524e92ad5722b1171fbef6/users/77b487fea3f94395823fc0855b86d085/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.615803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1373/2736] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:14 2022] PUT /identity/v3/projects/0ac06f0dea4f4496ba814e81d3b72cb8/users/547083e3258c4745a4c5d08db87942dc/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:14.627678 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4e868-eda0-401c-b0d3-02d0ce0148ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.628136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4e868-eda0-401c-b0d3-02d0ce0148ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.628503 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4e868-eda0-401c-b0d3-02d0ce0148ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.638059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdc1af-22a5-4318-868c-83dd58d865a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5cce08076524e92ad5722b1171fbef6, user_id=77b487fea3f94395823fc0855b86d085, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.640383 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdc1af-22a5-4318-868c-83dd58d865a1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.654959 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1364/2737] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:14 2022] PUT /identity/v3/projects/e5cce08076524e92ad5722b1171fbef6/users/77b487fea3f94395823fc0855b86d085/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:14.665607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fa8e-6782-41c1-a47a-ab72f9ed2b6e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.666299 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fa8e-6782-41c1-a47a-ab72f9ed2b6e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.666593 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fa8e-6782-41c1-a47a-ab72f9ed2b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.675147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c8a4e868-eda0-401c-b0d3-02d0ce0148ed None None] MFA Rules not processed for user `547083e3258c4745a4c5d08db87942dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:14.677107 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8a4e868-eda0-401c-b0d3-02d0ce0148ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:14.699828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c182fa8e-6782-41c1-a47a-ab72f9ed2b6e None None] MFA Rules not processed for user `77b487fea3f94395823fc0855b86d085`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:14.701059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c182fa8e-6782-41c1-a47a-ab72f9ed2b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:14.753233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1374/2738] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:14.758283 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4e868-eda0-401c-b0d3-02d0ce0148ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.773105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1365/2739] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:14.777574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fa8e-6782-41c1-a47a-ab72f9ed2b6e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.823772 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4e868-eda0-401c-b0d3-02d0ce0148ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.828526 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.829269 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.829673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.830053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.831202 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.833112 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.840562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fa8e-6782-41c1-a47a-ab72f9ed2b6e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.845135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.845766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.846042 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.846314 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.846932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.848840 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.859848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1375/2740] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/projects => generated 409 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:14.863891 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.882084 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1366/2741] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:14.883119 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.900924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:14.922877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:14.930080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a54af-7f7a-4054-a359-d811ac20a30c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.950125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43852f36-885b-4124-8ff7-b1634437ddf8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.954806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.956003 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.956276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.956541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.957156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.959046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:14.982109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1367/2742] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:14.987235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:14.993040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:14.993656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:14.993935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:14.994209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:14.994828 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:14.996589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698cb5a-d549-41ac-bab0-c41747fc37b8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:14.996865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.001061 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.001655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.001983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.002742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.003361 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.005071 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.015658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1368/2743] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:15.018517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1376/2744] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:14 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:15.019576 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.022297 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.029680 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d600f6-e841-40a0-842c-d637e3702892 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.033946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e8e34-9863-42f4-bf70-f8a67a9d27d9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.034463 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.035098 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.035396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.035892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.036534 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.038118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.038771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.039360 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.039640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.039946 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.040830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.042421 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.049499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1369/2745] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:15.053541 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.055862 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1377/2746] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:15.059585 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.063253 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cf122-9ea5-4cf5-96f3-b8fa48229b30 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.068412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905b109-2bc0-4fa3-815f-e2c4627b246f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.069065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b109-2bc0-4fa3-815f-e2c4627b246f admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.069339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b109-2bc0-4fa3-815f-e2c4627b246f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.069546 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2084-ea80-4177-bc7a-8f6a85f4781e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.069646 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905b109-2bc0-4fa3-815f-e2c4627b246f admin admin] PATH_INFO: `/v3/projects/a7e9e38cde5442b4966386ff66e0f0f6/users/c10c46bbf81640d38447e60530cfa7bf/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.074067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.074689 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.074966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.075240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] PATH_INFO: `/v3/projects/6b106c261a16452fb585604b180a8713/users/b75fcecaad994754ae6e243a055db48b/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.083667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905b109-2bc0-4fa3-815f-e2c4627b246f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e9e38cde5442b4966386ff66e0f0f6, user_id=c10c46bbf81640d38447e60530cfa7bf, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.086327 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905b109-2bc0-4fa3-815f-e2c4627b246f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.089597 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b106c261a16452fb585604b180a8713, user_id=b75fcecaad994754ae6e243a055db48b, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.092334 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.100034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1370/2747] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:15 2022] PUT /identity/v3/projects/a7e9e38cde5442b4966386ff66e0f0f6/users/c10c46bbf81640d38447e60530cfa7bf/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:15.109347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1378/2748] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:15 2022] PUT /identity/v3/projects/6b106c261a16452fb585604b180a8713/users/b75fcecaad994754ae6e243a055db48b/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:15.112787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073acf85-366d-47e9-8fcf-79e327ec311c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.113039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073acf85-366d-47e9-8fcf-79e327ec311c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.113286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073acf85-366d-47e9-8fcf-79e327ec311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.113970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.142151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-073acf85-366d-47e9-8fcf-79e327ec311c None None] MFA Rules not processed for user `c10c46bbf81640d38447e60530cfa7bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:15.143519 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-073acf85-366d-47e9-8fcf-79e327ec311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:15.179392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3e46c-52f3-434b-b3f8-413398ea6c42 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.184064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.184659 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.184935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.185204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.185859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.187433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.197785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1379/2749] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:15.201811 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.206289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1371/2750] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:15.210982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073acf85-366d-47e9-8fcf-79e327ec311c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.211253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b11cd-1f41-4137-bad1-66a63e1a50ef admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.215839 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6473f5b-35ce-4e8a-a88b-3842aac12e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.216653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6473f5b-35ce-4e8a-a88b-3842aac12e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.216653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6473f5b-35ce-4e8a-a88b-3842aac12e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.219413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6473f5b-35ce-4e8a-a88b-3842aac12e6b admin admin] PATH_INFO: `/v3/projects/6b106c261a16452fb585604b180a8713/users/b75fcecaad994754ae6e243a055db48b/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.219413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6473f5b-35ce-4e8a-a88b-3842aac12e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b106c261a16452fb585604b180a8713, user_id=b75fcecaad994754ae6e243a055db48b, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.221484 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6473f5b-35ce-4e8a-a88b-3842aac12e6b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.224971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073acf85-366d-47e9-8fcf-79e327ec311c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.229687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.230276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.230558 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.230830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.231450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.231552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1380/2751] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:15 2022] PUT /identity/v3/projects/6b106c261a16452fb585604b180a8713/users/b75fcecaad994754ae6e243a055db48b/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:15.233379 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.242212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.242496 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.242771 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.261900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1372/2752] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:15 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:15.265996 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.271147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] MFA Rules not processed for user `b75fcecaad994754ae6e243a055db48b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:15.272388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:15.295254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:15.304043 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:15.331245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98347f-7204-4c3d-980e-eea0bf95cb37 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.336201 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.336811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.337089 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.337380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.337987 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.339922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.346533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1381/2753] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:15 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:15.359867 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.361943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:15.367105 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1373/2754] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:15 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:15.368271 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.377847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e7a1b-fc13-4323-af26-169ec4a90c2e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.382430 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.383025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.383304 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.383575 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.384231 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.385843 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.396027 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1374/2755] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:15.399923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.411134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131e0b-b410-4f46-ad01-9b543c610751 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.416491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.417127 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.417406 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.417676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] PATH_INFO: `/v3/projects/8723d14bb8784af0988aebdd97be97aa/users/53175f0aeaaa41ac945a8a95f2e546dc/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.422325 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b0231-20c2-4465-a85a-4a74065dbd31 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.426734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.427337 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.427766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.428130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.433526 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8723d14bb8784af0988aebdd97be97aa, user_id=53175f0aeaaa41ac945a8a95f2e546dc, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.436194 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.437135 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.438570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.448606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1382/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:15.449059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1375/2757] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:15 2022] PUT /identity/v3/projects/8723d14bb8784af0988aebdd97be97aa/users/53175f0aeaaa41ac945a8a95f2e546dc/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:15.455978 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.466328 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8ddc9-28d4-4946-b28e-31ab3c696aff service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.528383 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.529082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.529369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.529643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.530276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.531884 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.545693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1383/2758] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:15.546891 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.558144 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de25fb9-c1f4-4590-b32f-c7a4a40a616e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.563053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f24b3-e4d1-4ec3-aeef-d0f7e9fc0e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.563712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f24b3-e4d1-4ec3-aeef-d0f7e9fc0e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.564032 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f24b3-e4d1-4ec3-aeef-d0f7e9fc0e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.564323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f24b3-e4d1-4ec3-aeef-d0f7e9fc0e3d admin admin] PATH_INFO: `/v3/projects/8723d14bb8784af0988aebdd97be97aa/users/53175f0aeaaa41ac945a8a95f2e546dc/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.566244 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f24b3-e4d1-4ec3-aeef-d0f7e9fc0e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8723d14bb8784af0988aebdd97be97aa, user_id=53175f0aeaaa41ac945a8a95f2e546dc, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.568915 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f24b3-e4d1-4ec3-aeef-d0f7e9fc0e3d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.579598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1376/2759] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:15 2022] PUT /identity/v3/projects/8723d14bb8784af0988aebdd97be97aa/users/53175f0aeaaa41ac945a8a95f2e546dc/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:15.615498 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:15.649252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.649450 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.649703 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.676242 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6536-6216-451e-83e5-91358c81e635 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:15.678480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] MFA Rules not processed for user `53175f0aeaaa41ac945a8a95f2e546dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:15.679991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:15.695343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:15.735937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:15.735937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:15.735937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:15.735937 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:15.766030 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1377/2760] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:15.795100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:15.796588 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:15.805706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1384/2761] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:16.001002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:16.002738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:16.017001 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44203c58-5df8-4c80-a4cf-cbbc0f8a0e8c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:16.021562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:16.022394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:16.022822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:16.023153 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:16.037319 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:16.038745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:16.066852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1378/2762] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:16.595281 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:16.603982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9079eb2-431f-4f5e-b7c5-156cba5fae34 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:16.607529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:16.608149 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:16.608429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:16.608700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:16.664320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:16.665992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:16.675244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1385/2763] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:24.924709 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:24.933165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f3657-766f-4198-807b-8f4126c4a596 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:24.936853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd1d38-5ceb-439b-b194-fc3faf08c7c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:24.937693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd1d38-5ceb-439b-b194-fc3faf08c7c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:24.937969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd1d38-5ceb-439b-b194-fc3faf08c7c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:24.938306 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd1d38-5ceb-439b-b194-fc3faf08c7c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:24.975821 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:24.994899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd1d38-5ceb-439b-b194-fc3faf08c7c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:24.996350 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd1d38-5ceb-439b-b194-fc3faf08c7c8 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:25.004905 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1379/2764] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:25.341452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:25.427300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e492d8-901d-42a6-ad7b-b68c77cb0746 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:25.441693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:25.442703 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:25.442989 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:25.443331 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] PATH_INFO: `/v3/users/547083e3258c4745a4c5d08db87942dc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:25.444716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=547083e3258c4745a4c5d08db87942dc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:25.447791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:25.465434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:25.506573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547083e3258c4745a4c5d08db87942dc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:25.515736 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547083e3258c4745a4c5d08db87942dc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:25.527903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547083e3258c4745a4c5d08db87942dc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:25.528595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547083e3258c4745a4c5d08db87942dc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:25.535847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547083e3258c4745a4c5d08db87942dc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:25.542436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1386/2765] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:25 2022] DELETE /identity/v3/users/547083e3258c4745a4c5d08db87942dc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:25.547021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:25.547309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:25.547582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:25.567585 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:25.568810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:25.634109 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1380/2766] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:25.644827 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:25.646206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:25.710997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff51f-4f49-4897-910d-567786728bcf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:25.715186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:25.715817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:25.716098 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:25.716370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:25.726654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:25.728072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:25.739808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1387/2767] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:26.014138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:26.015557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:26.030029 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab8ead-bebe-41bd-a44d-b7f6cde53df3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:26.035431 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:26.036269 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:26.036610 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:26.036963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] PATH_INFO: `/v3/projects/0ac06f0dea4f4496ba814e81d3b72cb8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:26.039397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac06f0dea4f4496ba814e81d3b72cb8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:26.042087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:26.087565 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac06f0dea4f4496ba814e81d3b72cb8', 'request_id': 'req-1501337e-426c-4b4f-949a-fd76ef750e2a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.092988 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac06f0dea4f4496ba814e81d3b72cb8', 'request_id': 'req-1501337e-426c-4b4f-949a-fd76ef750e2a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.101345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac06f0dea4f4496ba814e81d3b72cb8', 'request_id': 'req-1501337e-426c-4b4f-949a-fd76ef750e2a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.113421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] The token cache is being invalidate because project 0ac06f0dea4f4496ba814e81d3b72cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:26.113942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.114757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.116479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1381/2768] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:26 2022] DELETE /identity/v3/projects/0ac06f0dea4f4496ba814e81d3b72cb8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:26.121001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:26.130616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0fd28-9dda-4f91-8b1e-f0287772f71d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:26.187654 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:26.188306 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:26.188586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:26.188871 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] PATH_INFO: `/v3/users/b75fcecaad994754ae6e243a055db48b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:26.190024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75fcecaad994754ae6e243a055db48b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:26.191962 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:26.228701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:26.238731 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75fcecaad994754ae6e243a055db48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.245606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75fcecaad994754ae6e243a055db48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.257620 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75fcecaad994754ae6e243a055db48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.258256 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75fcecaad994754ae6e243a055db48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.265002 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f943470a-b321-440e-95ed-bd17c4e006a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75fcecaad994754ae6e243a055db48b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.271175 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1388/2769] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:26 2022] DELETE /identity/v3/users/b75fcecaad994754ae6e243a055db48b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:26.538320 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:26.540235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:26.553941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501337e-426c-4b4f-949a-fd76ef750e2a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:26.616021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:26.616656 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:26.616940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:26.617212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] PATH_INFO: `/v3/projects/6b106c261a16452fb585604b180a8713` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:26.618337 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b106c261a16452fb585604b180a8713)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:26.620305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:26.623043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:26.671642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b106c261a16452fb585604b180a8713', 'request_id': 'req-c214682c-fb15-41f8-9c27-35f1934d176e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.677433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b106c261a16452fb585604b180a8713', 'request_id': 'req-c214682c-fb15-41f8-9c27-35f1934d176e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.686051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b106c261a16452fb585604b180a8713', 'request_id': 'req-c214682c-fb15-41f8-9c27-35f1934d176e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.698272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] The token cache is being invalidate because project 6b106c261a16452fb585604b180a8713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:26.698563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.699206 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c214682c-fb15-41f8-9c27-35f1934d176e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:26.700733 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1382/2770] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:26 2022] DELETE /identity/v3/projects/6b106c261a16452fb585604b180a8713 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:26.706388 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd38ea1-754e-4cec-ae38-a2020f6e3787 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:26.706845 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd38ea1-754e-4cec-ae38-a2020f6e3787 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:26.706930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd38ea1-754e-4cec-ae38-a2020f6e3787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:26.727159 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ebd38ea1-754e-4cec-ae38-a2020f6e3787 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:26.728379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ebd38ea1-754e-4cec-ae38-a2020f6e3787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:26.794866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1389/2771] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:26.802242 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8dc9c-e799-42d2-8482-dbc0c854ea18 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:26.802537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8dc9c-e799-42d2-8482-dbc0c854ea18 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:26.802830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8dc9c-e799-42d2-8482-dbc0c854ea18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:26.826937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b9f8dc9c-e799-42d2-8482-dbc0c854ea18 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:26.828245 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b9f8dc9c-e799-42d2-8482-dbc0c854ea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:26.837289 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1383/2772] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:26.841567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd38ea1-754e-4cec-ae38-a2020f6e3787 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:26.851536 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd38ea1-754e-4cec-ae38-a2020f6e3787 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:26.856235 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:26.857055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:26.857523 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:26.857996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:26.859060 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:26.859903 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:26.861663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:26.876750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1390/2773] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 01:19:26.881659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8dc9c-e799-42d2-8482-dbc0c854ea18 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:26.891286 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8dc9c-e799-42d2-8482-dbc0c854ea18 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:26.902039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:26.902974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:26.903529 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:26.904497 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:26.905355 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:26.909913 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:26.951682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1384/2774] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:26 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:26.956528 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:26.967716 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640c6a1-e2c7-4cf0-a149-c233a39fef5b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.045028 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.045830 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.046110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.046382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.047016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.049114 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.072697 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1391/2775] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:26 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:27.076869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.088347 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e5d02-d4cd-4826-91f9-df0853035482 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.093065 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.093661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.093942 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.094212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.094878 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.096578 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.106648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1385/2776] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:27.111001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.122640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b5aad-9b86-47ea-96b7-ac199494f2ca admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.128257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122e7c7-4889-438b-9040-4d6d8843f175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.128859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122e7c7-4889-438b-9040-4d6d8843f175 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.129138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122e7c7-4889-438b-9040-4d6d8843f175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.129409 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122e7c7-4889-438b-9040-4d6d8843f175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.130081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122e7c7-4889-438b-9040-4d6d8843f175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.131652 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122e7c7-4889-438b-9040-4d6d8843f175 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.145379 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1392/2777] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:27.149714 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.163381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4717f8-e07e-4f6a-a35f-aaba0c2cf465 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.169598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.170408 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.170687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.170950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] PATH_INFO: `/v3/projects/ecfa01004b2d463393a153fd00af9271/users/fa1534b1a7fe4f909ea2dd8689eb0fcd/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.188025 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecfa01004b2d463393a153fd00af9271, user_id=fa1534b1a7fe4f909ea2dd8689eb0fcd, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.191465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.205474 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1386/2778] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:27 2022] PUT /identity/v3/projects/ecfa01004b2d463393a153fd00af9271/users/fa1534b1a7fe4f909ea2dd8689eb0fcd/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:27.217676 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792cd1f-d43a-4105-acfa-6c6a40f6bf02 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.218055 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792cd1f-d43a-4105-acfa-6c6a40f6bf02 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.218335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792cd1f-d43a-4105-acfa-6c6a40f6bf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.247912 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f792cd1f-d43a-4105-acfa-6c6a40f6bf02 None None] MFA Rules not processed for user `fa1534b1a7fe4f909ea2dd8689eb0fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:27.249373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f792cd1f-d43a-4105-acfa-6c6a40f6bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:27.259044 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:27.265760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.267555 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:27.328661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1393/2779] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:27.333357 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792cd1f-d43a-4105-acfa-6c6a40f6bf02 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.340090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7faf5-7ca4-4105-b0b9-2cd31fccbfa6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.343952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792cd1f-d43a-4105-acfa-6c6a40f6bf02 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.344801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.346645 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.347850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.347850 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] PATH_INFO: `/v3/users/77b487fea3f94395823fc0855b86d085` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.348486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b487fea3f94395823fc0855b86d085)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.350518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.351958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.352746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.353101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.353481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.354146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.356640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.389614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1394/2780] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:27 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:27.392225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:27.393932 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.409267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b487fea3f94395823fc0855b86d085'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:27.410410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b487fea3f94395823fc0855b86d085'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:27.422517 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b487fea3f94395823fc0855b86d085'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:27.437492 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b487fea3f94395823fc0855b86d085'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:27.438557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-30bf8e15-1cc4-40a1-8668-70d1906a64fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b487fea3f94395823fc0855b86d085'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:27.451432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1387/2781] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:27 2022] DELETE /identity/v3/users/77b487fea3f94395823fc0855b86d085 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:27.455721 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26633f5c-610e-4825-901c-043965942df0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.456037 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26633f5c-610e-4825-901c-043965942df0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.456317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26633f5c-610e-4825-901c-043965942df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.470661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8462aa-3d37-4372-9717-ae1bee4304c9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.478411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-26633f5c-610e-4825-901c-043965942df0 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:27.479636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-26633f5c-610e-4825-901c-043965942df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:27.532959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.533573 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.534370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.534644 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.535277 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.537214 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.544690 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1388/2782] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:27.555514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26633f5c-610e-4825-901c-043965942df0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.556912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-26633f5c-610e-4825-901c-043965942df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:27.560053 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1395/2783] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:27 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:27.564619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.566693 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:27.579273 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf8408-e9a7-4854-a808-929eb0687d61 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.584774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.585380 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.585766 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.585910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.586548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.588162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.598769 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1396/2784] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:27.602799 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.613014 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992d9df-66f1-4d7b-8b04-ac1ffcae792e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.618698 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.619321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.619721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.620012 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.620725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.622558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.623938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26633f5c-610e-4825-901c-043965942df0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.627587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b615c5-9ef2-49d0-9210-92d667badeec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.628225 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b615c5-9ef2-49d0-9210-92d667badeec service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.628501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b615c5-9ef2-49d0-9210-92d667badeec service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.628768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b615c5-9ef2-49d0-9210-92d667badeec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.634136 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1397/2785] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:27.638235 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b615c5-9ef2-49d0-9210-92d667badeec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.638371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.639655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b615c5-9ef2-49d0-9210-92d667badeec service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.643104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:27.649351 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568e8f9-e3b9-4020-b321-0c04242199ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.649899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1389/2786] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:27.656034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.656696 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.656977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.657255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] PATH_INFO: `/v3/projects/2c2fa95b719147bca463d90691270706/users/e825b891cb174700ad2dc66eca557d1c/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.675373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2fa95b719147bca463d90691270706, user_id=e825b891cb174700ad2dc66eca557d1c, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.678120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.691846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1398/2787] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:27 2022] PUT /identity/v3/projects/2c2fa95b719147bca463d90691270706/users/e825b891cb174700ad2dc66eca557d1c/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:27.702480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d9bf5-1c4e-4e19-9003-c06c1fcb9a6e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.702768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d9bf5-1c4e-4e19-9003-c06c1fcb9a6e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.703043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d9bf5-1c4e-4e19-9003-c06c1fcb9a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.742998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ff2d9bf5-1c4e-4e19-9003-c06c1fcb9a6e None None] MFA Rules not processed for user `e825b891cb174700ad2dc66eca557d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:27.744156 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ff2d9bf5-1c4e-4e19-9003-c06c1fcb9a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:27.790324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:27.821939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1390/2788] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:27.826392 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.895369 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d057cc-a38f-4ea1-b3c3-57c137b6ebd4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.900158 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:27.900762 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:27.901041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:27.901314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:27.901938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:27.903880 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:27.932850 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1399/2789] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:27 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:27.937005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d9bf5-1c4e-4e19-9003-c06c1fcb9a6e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.948717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d9bf5-1c4e-4e19-9003-c06c1fcb9a6e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:27.963504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:27.969732 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.036992 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.037704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.037967 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.038478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.040356 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.042811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.071967 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ead17-7f5e-458a-a5bd-dc0c8effc265 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.072106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1391/2790] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:27 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:28.072671 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.074719 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.075359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.075669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.076101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] PATH_INFO: `/v3/projects/e5cce08076524e92ad5722b1171fbef6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.077319 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5cce08076524e92ad5722b1171fbef6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.079456 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.084941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bddff-fff4-4868-a1b3-326f23c96446 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.090950 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.091568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.091991 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.092270 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.092920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.094554 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.105738 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1392/2791] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:28.109892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.120954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b191a21-a6e7-4048-8d10-aa4441721edb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.140239 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5cce08076524e92ad5722b1171fbef6', 'request_id': 'req-b810333f-5c33-44b8-8a6d-7bc9a4da026a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.147744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5cce08076524e92ad5722b1171fbef6', 'request_id': 'req-b810333f-5c33-44b8-8a6d-7bc9a4da026a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.155734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5cce08076524e92ad5722b1171fbef6', 'request_id': 'req-b810333f-5c33-44b8-8a6d-7bc9a4da026a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.157233 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:28.170130 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] The token cache is being invalidate because project e5cce08076524e92ad5722b1171fbef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:28.170423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.171080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.172855 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1400/2792] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:27 2022] DELETE /identity/v3/projects/e5cce08076524e92ad5722b1171fbef6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:28.176525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.177894 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.185582 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.186424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.186707 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.186983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] PATH_INFO: `/v3/projects/7d4815fe11a241619447c044b65ae15b/users/1201e397602d4208920324f7735c181a/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.205310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4815fe11a241619447c044b65ae15b, user_id=1201e397602d4208920324f7735c181a, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.208611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.222072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1393/2793] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:28 2022] PUT /identity/v3/projects/7d4815fe11a241619447c044b65ae15b/users/1201e397602d4208920324f7735c181a/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:28.226093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.227936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.250096 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810333f-5c33-44b8-8a6d-7bc9a4da026a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.300086 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:28.320758 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.321405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.321688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.321966 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] PATH_INFO: `/v3/users/c10c46bbf81640d38447e60530cfa7bf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.323163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10c46bbf81640d38447e60530cfa7bf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.323364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba08ae-042a-4540-8cdb-7bef4c91d117 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.325234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.328252 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.328857 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.329141 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.329411 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.330051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.332313 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.343916 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1394/2794] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:28.347333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.358090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c262-8672-48f3-b4d2-bf46c42aaab4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.364426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef66ab7-4238-409c-bf31-86c78986c290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.367093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef66ab7-4238-409c-bf31-86c78986c290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.368533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef66ab7-4238-409c-bf31-86c78986c290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.368813 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef66ab7-4238-409c-bf31-86c78986c290 admin admin] PATH_INFO: `/v3/projects/7d4815fe11a241619447c044b65ae15b/users/1201e397602d4208920324f7735c181a/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.371177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef66ab7-4238-409c-bf31-86c78986c290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4815fe11a241619447c044b65ae15b, user_id=1201e397602d4208920324f7735c181a, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.374126 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef66ab7-4238-409c-bf31-86c78986c290 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.387740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1395/2795] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:28 2022] PUT /identity/v3/projects/7d4815fe11a241619447c044b65ae15b/users/1201e397602d4208920324f7735c181a/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:28.400720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.401013 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.401293 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.401529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10c46bbf81640d38447e60530cfa7bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.415506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10c46bbf81640d38447e60530cfa7bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.437612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] MFA Rules not processed for user `1201e397602d4208920324f7735c181a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:28.438900 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.440754 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10c46bbf81640d38447e60530cfa7bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.441566 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10c46bbf81640d38447e60530cfa7bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.449168 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10c46bbf81640d38447e60530cfa7bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.455381 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1401/2796] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:28 2022] DELETE /identity/v3/users/c10c46bbf81640d38447e60530cfa7bf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:28.526989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1396/2797] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:28.543073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.545147 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.571882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:28.621143 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc956a-b981-41b1-8c89-62d1f644ce30 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.625493 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.626163 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.626451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.626859 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.697509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.698994 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.709562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1402/2798] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:28.759417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.761214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.776847 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.778991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.843969 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76879a7b-c04f-4df2-8ade-2429d05dba27 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.849135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.849755 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.850067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.850354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.852177 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:28.854428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd703d02-69d3-42ee-a466-86367c579319 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:28.859081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:28.859709 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:28.860019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:28.860294 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] PATH_INFO: `/v3/projects/a7e9e38cde5442b4966386ff66e0f0f6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:28.861504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e9e38cde5442b4966386ff66e0f0f6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.863430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.914637 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:28.916397 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:28.920142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e9e38cde5442b4966386ff66e0f0f6', 'request_id': 'req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.925479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1397/2799] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:28.925687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e9e38cde5442b4966386ff66e0f0f6', 'request_id': 'req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.940866 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e9e38cde5442b4966386ff66e0f0f6', 'request_id': 'req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.949938 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:28.955226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] The token cache is being invalidate because project a7e9e38cde5442b4966386ff66e0f0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:28.956310 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.957371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:28.959635 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1403/2800] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:28 2022] DELETE /identity/v3/projects/a7e9e38cde5442b4966386ff66e0f0f6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:28.963504 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:28.972068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:28.976490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f4bf7-6b76-4417-9da7-e17dd51636b0 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:29.040692 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:29.041553 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.042111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.042544 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] PATH_INFO: `/v3/users/53175f0aeaaa41ac945a8a95f2e546dc` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.046434 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53175f0aeaaa41ac945a8a95f2e546dc)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:29.048611 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:29.102677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53175f0aeaaa41ac945a8a95f2e546dc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.103385 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53175f0aeaaa41ac945a8a95f2e546dc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.112157 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53175f0aeaaa41ac945a8a95f2e546dc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.133625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53175f0aeaaa41ac945a8a95f2e546dc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.133625 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2f65883b-42cb-4535-965c-fa1a57703a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53175f0aeaaa41ac945a8a95f2e546dc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.146767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1398/2801] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:28 2022] DELETE /identity/v3/users/53175f0aeaaa41ac945a8a95f2e546dc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:29.469105 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:29.470601 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:29.489161 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c35e-8335-4f54-a11b-88cfd9ab45c9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:29.549606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:29.550209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.550491 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.550764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] PATH_INFO: `/v3/projects/8723d14bb8784af0988aebdd97be97aa` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.551939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8723d14bb8784af0988aebdd97be97aa)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:29.553843 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:29.604302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8723d14bb8784af0988aebdd97be97aa', 'request_id': 'req-37c241c0-64ee-49ea-a182-38ff0c670834'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.610022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8723d14bb8784af0988aebdd97be97aa', 'request_id': 'req-37c241c0-64ee-49ea-a182-38ff0c670834'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.618091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8723d14bb8784af0988aebdd97be97aa', 'request_id': 'req-37c241c0-64ee-49ea-a182-38ff0c670834'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.629675 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] The token cache is being invalidate because project 8723d14bb8784af0988aebdd97be97aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:29.629963 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.630582 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37c241c0-64ee-49ea-a182-38ff0c670834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:29.632087 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1404/2802] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:29 2022] DELETE /identity/v3/projects/8723d14bb8784af0988aebdd97be97aa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:29.638123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422561d2-36c3-43da-8295-5f6d6ca2dd4d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.638415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422561d2-36c3-43da-8295-5f6d6ca2dd4d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.638691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422561d2-36c3-43da-8295-5f6d6ca2dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.657953 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-422561d2-36c3-43da-8295-5f6d6ca2dd4d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:29.659367 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-422561d2-36c3-43da-8295-5f6d6ca2dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:29.720273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1399/2803] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:29.728666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa017a-11fd-42e6-9761-b82cbeaf5437 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.728954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa017a-11fd-42e6-9761-b82cbeaf5437 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.729234 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa017a-11fd-42e6-9761-b82cbeaf5437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.749245 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c6fa017a-11fd-42e6-9761-b82cbeaf5437 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:29.750452 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6fa017a-11fd-42e6-9761-b82cbeaf5437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:29.759129 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1405/2804] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:29.763455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422561d2-36c3-43da-8295-5f6d6ca2dd4d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:29.774241 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422561d2-36c3-43da-8295-5f6d6ca2dd4d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:29.778917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:29.779550 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.779871 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.780149 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.780789 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:29.781093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:29.782719 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:29.794449 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1400/2805] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:29.798803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa017a-11fd-42e6-9761-b82cbeaf5437 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:29.808550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa017a-11fd-42e6-9761-b82cbeaf5437 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:29.813999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:29.814661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.815596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.815955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.816673 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:29.818749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:29.845885 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1406/2806] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:29 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:29.850147 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:29.861048 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e8cc9-0859-4421-a7f8-385e42d54656 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:29.872181 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:29.921305 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:29.921963 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.922273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.922577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.923264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:29.925766 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:29.947391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1401/2807] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:29 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:29.951483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:29.961768 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51a91-86eb-4f2e-93a4-7a90272bb15a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:29.966402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:29.967001 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:29.967279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:29.967549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:29.968225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:29.969832 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:29.980701 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1407/2808] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:29.984851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:29.995403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9a961-05a4-4e30-9a74-bec771aa0fd7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:30.000329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca022fb7-bc0b-49d5-b8f2-337f030bd022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:30.000934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca022fb7-bc0b-49d5-b8f2-337f030bd022 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:30.001214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca022fb7-bc0b-49d5-b8f2-337f030bd022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:30.001486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca022fb7-bc0b-49d5-b8f2-337f030bd022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:30.002161 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca022fb7-bc0b-49d5-b8f2-337f030bd022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:30.003832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca022fb7-bc0b-49d5-b8f2-337f030bd022 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:30.014615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1402/2809] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:30.019426 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:30.030895 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714baad-6492-4936-9fea-5ff3c5844093 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:30.036596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:30.037448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:30.037725 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:30.037991 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] PATH_INFO: `/v3/projects/55e513e86e3742aa9e9f77597f3599c6/users/df622011fc604f37923fe02f0960ea4d/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:30.054209 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e513e86e3742aa9e9f77597f3599c6, user_id=df622011fc604f37923fe02f0960ea4d, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:30.056883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:30.069893 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1408/2810] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:30 2022] PUT /identity/v3/projects/55e513e86e3742aa9e9f77597f3599c6/users/df622011fc604f37923fe02f0960ea4d/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:30.082879 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:30.083164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:30.083439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:30.110715 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] MFA Rules not processed for user `df622011fc604f37923fe02f0960ea4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:30.111947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:30.179240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1403/2811] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:30.193399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:30.195179 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:30.227977 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:30.257870 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2a4ab-eafd-4dd5-833c-2534f47d0e7e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:30.261563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4ac7a-5f67-4106-8eb0-c79768fb60e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:30.262170 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4ac7a-5f67-4106-8eb0-c79768fb60e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:30.262447 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4ac7a-5f67-4106-8eb0-c79768fb60e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:30.262721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4ac7a-5f67-4106-8eb0-c79768fb60e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:30.270888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4ac7a-5f67-4106-8eb0-c79768fb60e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:30.272343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4ac7a-5f67-4106-8eb0-c79768fb60e6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:30.281145 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1409/2812] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:31.925890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:31.927366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:32.008806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:32.032846 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a1906-c57e-40ed-81d1-52829f402c67 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:32.050934 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:32.051559 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:32.051877 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:32.052151 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] PATH_INFO: `/v3/users/fa1534b1a7fe4f909ea2dd8689eb0fcd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:32.053266 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1534b1a7fe4f909ea2dd8689eb0fcd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:32.055170 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:32.102370 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1534b1a7fe4f909ea2dd8689eb0fcd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.102948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1534b1a7fe4f909ea2dd8689eb0fcd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.111058 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1534b1a7fe4f909ea2dd8689eb0fcd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.123855 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1534b1a7fe4f909ea2dd8689eb0fcd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.124493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1534b1a7fe4f909ea2dd8689eb0fcd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.136326 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1404/2813] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:31 2022] DELETE /identity/v3/users/fa1534b1a7fe4f909ea2dd8689eb0fcd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:32.141320 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:32.141865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:32.141959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:32.170572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:32.171776 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:32.243248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1410/2814] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:32.254358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:32.255852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:32.337318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a63dd-e31a-4512-b54a-a55a0f980359 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:32.341093 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:32.341700 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:32.341982 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:32.342256 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:32.351329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:32.352865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:32.362046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1405/2815] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:32.636549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:32.638035 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:32.650533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1b9f8-a70b-4d29-99f3-6b5ec3c2ee7a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:32.655817 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:32.656502 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:32.656804 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:32.657563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] PATH_INFO: `/v3/projects/ecfa01004b2d463393a153fd00af9271` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:32.658344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecfa01004b2d463393a153fd00af9271)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:32.662473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:32.714694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecfa01004b2d463393a153fd00af9271', 'request_id': 'req-a8dbd58a-7c86-42f3-a381-01fc4053560c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.720370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecfa01004b2d463393a153fd00af9271', 'request_id': 'req-a8dbd58a-7c86-42f3-a381-01fc4053560c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.728095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecfa01004b2d463393a153fd00af9271', 'request_id': 'req-a8dbd58a-7c86-42f3-a381-01fc4053560c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.739648 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] The token cache is being invalidate because project ecfa01004b2d463393a153fd00af9271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:32.740133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.740848 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.742506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1411/2816] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:32 2022] DELETE /identity/v3/projects/ecfa01004b2d463393a153fd00af9271 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:19:32.747478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:32.760043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1247da-c0d6-4853-aa22-850096cdc3d8 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:32.791031 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:32.820112 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:32.820734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:32.821008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:32.821273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] PATH_INFO: `/v3/users/e825b891cb174700ad2dc66eca557d1c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:32.822354 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e825b891cb174700ad2dc66eca557d1c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:32.824272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:32.869190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e825b891cb174700ad2dc66eca557d1c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.869851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e825b891cb174700ad2dc66eca557d1c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.877295 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e825b891cb174700ad2dc66eca557d1c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.890410 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e825b891cb174700ad2dc66eca557d1c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.891011 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e825b891cb174700ad2dc66eca557d1c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:32.902922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1406/2817] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:32 2022] DELETE /identity/v3/users/e825b891cb174700ad2dc66eca557d1c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:33.174944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:33.176990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:33.190102 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd58a-7c86-42f3-a381-01fc4053560c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:33.222505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:33.251215 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:33.251881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:33.252161 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:33.252433 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] PATH_INFO: `/v3/projects/2c2fa95b719147bca463d90691270706` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:33.253559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2fa95b719147bca463d90691270706)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:33.255443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:33.303646 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2fa95b719147bca463d90691270706', 'request_id': 'req-29ea36d6-dc29-4953-bd66-2273a250d28e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.309419 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2fa95b719147bca463d90691270706', 'request_id': 'req-29ea36d6-dc29-4953-bd66-2273a250d28e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.317362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2fa95b719147bca463d90691270706', 'request_id': 'req-29ea36d6-dc29-4953-bd66-2273a250d28e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.330559 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] The token cache is being invalidate because project 2c2fa95b719147bca463d90691270706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:33.330881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.331586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.333935 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1412/2818] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:33 2022] DELETE /identity/v3/projects/2c2fa95b719147bca463d90691270706 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:33.337923 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:33.339423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:33.353118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e1376-1045-4d1b-b316-a5264ee77de5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:33.409264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:33.409887 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:33.410162 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:33.410989 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] PATH_INFO: `/v3/users/1201e397602d4208920324f7735c181a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:33.412123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1201e397602d4208920324f7735c181a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:33.414023 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:33.460631 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1201e397602d4208920324f7735c181a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.461265 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1201e397602d4208920324f7735c181a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.468424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1201e397602d4208920324f7735c181a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.469507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:33.482640 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1201e397602d4208920324f7735c181a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.483321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6855c365-6381-439e-9ea2-5352975ea60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1201e397602d4208920324f7735c181a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.500997 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1407/2819] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:33 2022] DELETE /identity/v3/users/1201e397602d4208920324f7735c181a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:33.783497 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:33.784943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:33.798211 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea36d6-dc29-4953-bd66-2273a250d28e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:33.858178 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:33.858805 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:33.859083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:33.859355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] PATH_INFO: `/v3/projects/7d4815fe11a241619447c044b65ae15b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:33.860506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4815fe11a241619447c044b65ae15b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:33.862377 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:33.914663 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4815fe11a241619447c044b65ae15b', 'request_id': 'req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.920680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4815fe11a241619447c044b65ae15b', 'request_id': 'req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.929511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4815fe11a241619447c044b65ae15b', 'request_id': 'req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.930856 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:33.944563 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] The token cache is being invalidate because project 7d4815fe11a241619447c044b65ae15b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:33.944857 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.945512 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3cd0ef5-9d23-4112-87dc-f34ef0367b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:33.946999 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1413/2820] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:33 2022] DELETE /identity/v3/projects/7d4815fe11a241619447c044b65ae15b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:33.953886 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e251e-957b-46b9-b9f4-e701946000ea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:33.954150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e251e-957b-46b9-b9f4-e701946000ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:33.954439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e251e-957b-46b9-b9f4-e701946000ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.003291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ad3e251e-957b-46b9-b9f4-e701946000ea None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:34.004453 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad3e251e-957b-46b9-b9f4-e701946000ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:34.063264 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1408/2821] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:34.072714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e28a92-68b0-44b2-bc7b-5e14e49b69d9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.073067 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e28a92-68b0-44b2-bc7b-5e14e49b69d9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.073410 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e28a92-68b0-44b2-bc7b-5e14e49b69d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.093722 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a8e28a92-68b0-44b2-bc7b-5e14e49b69d9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:34.094914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8e28a92-68b0-44b2-bc7b-5e14e49b69d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:34.103122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1414/2822] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:34.107454 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e251e-957b-46b9-b9f4-e701946000ea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:34.117495 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e251e-957b-46b9-b9f4-e701946000ea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:34.122740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:34.123362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.123636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.123935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.124735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:34.125138 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:34.126932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:34.139143 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1409/2823] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:34.143666 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e28a92-68b0-44b2-bc7b-5e14e49b69d9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:34.153251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e28a92-68b0-44b2-bc7b-5e14e49b69d9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:34.157872 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:34.158468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.158746 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.159017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.159637 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:34.162148 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:34.191531 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1415/2824] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:34 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:34.195072 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:34.204787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06331b-048d-48ad-ac43-fe8b6bd9741b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:34.262551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:34.263184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.263468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.263742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.264445 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:34.266380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:34.291439 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1410/2825] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:34 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:34.295556 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:34.305908 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe6075-964a-4e33-8a05-d5de9dc6ea01 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:34.310488 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:34.311080 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.311362 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.311636 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.312301 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:34.313876 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:34.323567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1416/2826] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:34.327551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:34.338712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018edae-1d63-464e-a483-3db79989f630 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:34.343365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0daab-1337-4b30-a5b5-332345bcc5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:34.344005 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0daab-1337-4b30-a5b5-332345bcc5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.344284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0daab-1337-4b30-a5b5-332345bcc5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.344556 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0daab-1337-4b30-a5b5-332345bcc5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.345240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0daab-1337-4b30-a5b5-332345bcc5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:34.346827 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0daab-1337-4b30-a5b5-332345bcc5e9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:34.358423 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1411/2827] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:34.362442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:34.373457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9f2b6-7da9-4299-8977-d060e2291f4f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:34.381948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:34.382705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.383052 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.383367 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] PATH_INFO: `/v3/projects/94cb1ab2700449bd9057360cebaaf0ab/users/1e403888ea2d4aef8fe9334a32ca0a1a/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.403451 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cb1ab2700449bd9057360cebaaf0ab, user_id=1e403888ea2d4aef8fe9334a32ca0a1a, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:34.406146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:34.420076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1417/2828] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:34 2022] PUT /identity/v3/projects/94cb1ab2700449bd9057360cebaaf0ab/users/1e403888ea2d4aef8fe9334a32ca0a1a/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:34.431744 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119d876-956f-4002-acc8-d640184a2a2f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.432043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119d876-956f-4002-acc8-d640184a2a2f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.432317 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119d876-956f-4002-acc8-d640184a2a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.458116 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:34.461608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8119d876-956f-4002-acc8-d640184a2a2f None None] MFA Rules not processed for user `1e403888ea2d4aef8fe9334a32ca0a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:34.463421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8119d876-956f-4002-acc8-d640184a2a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:34.529110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1412/2829] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:34.540920 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:34.542606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:34.604417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b693e59-f51b-41c9-ab34-d8853b56873f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:34.608082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d9262-2928-4bcb-bdab-184e63498db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:34.608694 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d9262-2928-4bcb-bdab-184e63498db5 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:34.608952 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d9262-2928-4bcb-bdab-184e63498db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:34.609228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d9262-2928-4bcb-bdab-184e63498db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:34.618864 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d9262-2928-4bcb-bdab-184e63498db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:34.620413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d9262-2928-4bcb-bdab-184e63498db5 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:34.629587 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1418/2830] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:19:40.389184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119d876-956f-4002-acc8-d640184a2a2f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:40.460412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119d876-956f-4002-acc8-d640184a2a2f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:40.465123 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:40.465782 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:40.466080 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:40.466349 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] PATH_INFO: `/v3/users/1e403888ea2d4aef8fe9334a32ca0a1a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:40.467451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e403888ea2d4aef8fe9334a32ca0a1a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:40.469394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:40.514524 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e403888ea2d4aef8fe9334a32ca0a1a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:40.515134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e403888ea2d4aef8fe9334a32ca0a1a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:40.523338 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e403888ea2d4aef8fe9334a32ca0a1a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:40.536329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e403888ea2d4aef8fe9334a32ca0a1a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:40.536983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e403888ea2d4aef8fe9334a32ca0a1a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:40.549277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1413/2831] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:40 2022] DELETE /identity/v3/users/1e403888ea2d4aef8fe9334a32ca0a1a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:40.553945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:40.554435 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:40.554567 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:40.575430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:40.576664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:40.636860 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1419/2832] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:19:40.647075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:40.648479 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:40.677781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:40.712907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f5ccd-bca7-42ca-a4ab-f6e5a76d4ae2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:40.716545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69676bd1-0e4a-4289-8263-5f091134f861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:40.717148 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69676bd1-0e4a-4289-8263-5f091134f861 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:40.717551 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69676bd1-0e4a-4289-8263-5f091134f861 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:40.717787 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69676bd1-0e4a-4289-8263-5f091134f861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:40.729254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69676bd1-0e4a-4289-8263-5f091134f861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:40.730720 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69676bd1-0e4a-4289-8263-5f091134f861 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:40.740484 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1414/2833] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:41.033068 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.034505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:41.047133 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514ba5e-241d-41ae-a1ad-aad481fe316b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.052436 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.053072 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.053352 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.053627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] PATH_INFO: `/v3/projects/94cb1ab2700449bd9057360cebaaf0ab` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.054742 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cb1ab2700449bd9057360cebaaf0ab)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.056683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.094064 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:41.110786 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cb1ab2700449bd9057360cebaaf0ab', 'request_id': 'req-04b70f69-3457-4481-aae7-b76ee58f0e59'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:41.118370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cb1ab2700449bd9057360cebaaf0ab', 'request_id': 'req-04b70f69-3457-4481-aae7-b76ee58f0e59'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:41.127434 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cb1ab2700449bd9057360cebaaf0ab', 'request_id': 'req-04b70f69-3457-4481-aae7-b76ee58f0e59'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:41.142449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] The token cache is being invalidate because project 94cb1ab2700449bd9057360cebaaf0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:41.142743 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:41.143365 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-04b70f69-3457-4481-aae7-b76ee58f0e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:41.144909 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1420/2834] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:41 2022] DELETE /identity/v3/projects/94cb1ab2700449bd9057360cebaaf0ab => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:41.149712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adbb8f-abcb-434a-8e8d-81a670a89f29 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.149998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adbb8f-abcb-434a-8e8d-81a670a89f29 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.150272 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adbb8f-abcb-434a-8e8d-81a670a89f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.170044 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-08adbb8f-abcb-434a-8e8d-81a670a89f29 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:41.171234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-08adbb8f-abcb-434a-8e8d-81a670a89f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:41.228412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1415/2835] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:41.292309 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b396de-773f-4706-9d6a-0a4c97784866 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.292600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b396de-773f-4706-9d6a-0a4c97784866 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.292879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b396de-773f-4706-9d6a-0a4c97784866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.314695 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-04b396de-773f-4706-9d6a-0a4c97784866 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:41.315953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04b396de-773f-4706-9d6a-0a4c97784866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:41.325083 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1421/2836] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:41.329334 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adbb8f-abcb-434a-8e8d-81a670a89f29 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.339075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adbb8f-abcb-434a-8e8d-81a670a89f29 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.344008 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.344796 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.345078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.345353 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.346039 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:41.346374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.348763 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.360896 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1416/2837] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:41.368299 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b396de-773f-4706-9d6a-0a4c97784866 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.377574 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b396de-773f-4706-9d6a-0a4c97784866 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.382428 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.383033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.383314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.383591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.384254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.386146 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.422444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1422/2838] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:41 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:41.423276 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.436954 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312081-b6e8-4aea-bf08-8b9bb1a8a2a6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.512435 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.513051 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.513332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.513603 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.514273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.516228 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.544708 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1417/2839] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:41 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:19:41.546015 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.559778 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ad1c0-76fa-4449-aa41-f70d891b1044 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.566212 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.566975 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.567322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.567615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.568568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.570444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.587882 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1423/2840] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:41.593968 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.606863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b47f9a-cceb-4f1c-9798-c78cb04e31a2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.612362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19978ba1-5afa-48cc-a996-bd503d582577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.613050 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19978ba1-5afa-48cc-a996-bd503d582577 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.613572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19978ba1-5afa-48cc-a996-bd503d582577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.614481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19978ba1-5afa-48cc-a996-bd503d582577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.614817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19978ba1-5afa-48cc-a996-bd503d582577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.616318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19978ba1-5afa-48cc-a996-bd503d582577 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.631020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1418/2841] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:41.632271 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.644046 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e745c3-ffd2-47fd-b0a7-dbc87171934f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.649774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.650681 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.650974 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.651248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] PATH_INFO: `/v3/projects/1542fa8303a74df98d1569303c6c2edc/users/e181f371086e4b48972314529d30d90c/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.670378 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1542fa8303a74df98d1569303c6c2edc, user_id=e181f371086e4b48972314529d30d90c, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.673259 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.688373 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1424/2842] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:41 2022] PUT /identity/v3/projects/1542fa8303a74df98d1569303c6c2edc/users/e181f371086e4b48972314529d30d90c/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:41.699311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f5534-1b0f-4d53-9372-6a4a710267b5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.699805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f5534-1b0f-4d53-9372-6a4a710267b5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.700311 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f5534-1b0f-4d53-9372-6a4a710267b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.736870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-965f5534-1b0f-4d53-9372-6a4a710267b5 None None] MFA Rules not processed for user `e181f371086e4b48972314529d30d90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:41.738518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-965f5534-1b0f-4d53-9372-6a4a710267b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:41.763466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:41.842279 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1419/2843] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:41.850658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:41.854332 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:41.933470 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c418c9-8be5-435f-8c02-46d9dfbc7b9b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:41.938340 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212725b-8ee9-46c7-bf14-bbc96491cb7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:41.939016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212725b-8ee9-46c7-bf14-bbc96491cb7b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:41.939296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212725b-8ee9-46c7-bf14-bbc96491cb7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:41.939570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212725b-8ee9-46c7-bf14-bbc96491cb7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:41.948888 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212725b-8ee9-46c7-bf14-bbc96491cb7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:41.950300 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212725b-8ee9-46c7-bf14-bbc96491cb7b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:41.959803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1425/2844] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:41.985685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f5534-1b0f-4d53-9372-6a4a710267b5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:42.047685 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f5534-1b0f-4d53-9372-6a4a710267b5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:42.052199 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:42.052812 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.053088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.053362 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] PATH_INFO: `/v3/users/e181f371086e4b48972314529d30d90c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.054458 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] RBAC: Authorizing `identity:delete_user(user_id=e181f371086e4b48972314529d30d90c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:42.056725 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:42.105019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e181f371086e4b48972314529d30d90c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.105661 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e181f371086e4b48972314529d30d90c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.112881 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e181f371086e4b48972314529d30d90c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.126641 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e181f371086e4b48972314529d30d90c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.127239 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e181f371086e4b48972314529d30d90c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.140802 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1420/2845] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:41 2022] DELETE /identity/v3/users/e181f371086e4b48972314529d30d90c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:42.145984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.146255 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.146527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.165773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:42.166982 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:42.213704 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:42.225990 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1426/2846] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:42.235568 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:42.237100 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:42.280432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:42.295368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fbeb1-1aad-46ca-a3f1-a4c7508624df admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:42.299014 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d29fdf-6e57-49ce-a604-f17cef3dbb33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:42.299612 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d29fdf-6e57-49ce-a604-f17cef3dbb33 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.299926 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d29fdf-6e57-49ce-a604-f17cef3dbb33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.300202 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d29fdf-6e57-49ce-a604-f17cef3dbb33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.310436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d29fdf-6e57-49ce-a604-f17cef3dbb33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:42.311893 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d29fdf-6e57-49ce-a604-f17cef3dbb33 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:42.321663 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1421/2847] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:42.594223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:42.595813 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:42.608314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a2788-520b-43a1-9c53-63283c4964fa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:42.613113 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:42.613753 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.614033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.614303 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] PATH_INFO: `/v3/projects/1542fa8303a74df98d1569303c6c2edc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.615417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1542fa8303a74df98d1569303c6c2edc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:42.617363 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:42.665150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1542fa8303a74df98d1569303c6c2edc', 'request_id': 'req-74129377-ad6f-4df7-8a41-851f88760882'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.670788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1542fa8303a74df98d1569303c6c2edc', 'request_id': 'req-74129377-ad6f-4df7-8a41-851f88760882'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.677720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1542fa8303a74df98d1569303c6c2edc', 'request_id': 'req-74129377-ad6f-4df7-8a41-851f88760882'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.688521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] The token cache is being invalidate because project 1542fa8303a74df98d1569303c6c2edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:42.688818 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.689445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74129377-ad6f-4df7-8a41-851f88760882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:42.690961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1427/2848] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:42 2022] DELETE /identity/v3/projects/1542fa8303a74df98d1569303c6c2edc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:42.696503 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c1ad-43a6-4478-9815-c75a4b079d5f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.696972 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c1ad-43a6-4478-9815-c75a4b079d5f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.697088 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c1ad-43a6-4478-9815-c75a4b079d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.716956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a1c4c1ad-43a6-4478-9815-c75a4b079d5f None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:42.718159 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a1c4c1ad-43a6-4478-9815-c75a4b079d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:42.780815 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1422/2849] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:42.788910 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ca351-7107-4fc7-a668-0588773c7820 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.789201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ca351-7107-4fc7-a668-0588773c7820 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.789481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ca351-7107-4fc7-a668-0588773c7820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.809533 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-551ca351-7107-4fc7-a668-0588773c7820 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:42.810744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-551ca351-7107-4fc7-a668-0588773c7820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:42.819137 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1428/2850] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:42.823460 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c1ad-43a6-4478-9815-c75a4b079d5f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:42.834094 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c1ad-43a6-4478-9815-c75a4b079d5f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:42.838702 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:42.839301 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.839606 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.839914 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.840557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:42.840861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:42.842493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:42.854432 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1423/2851] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:42.858802 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ca351-7107-4fc7-a668-0588773c7820 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:42.868223 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ca351-7107-4fc7-a668-0588773c7820 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:42.873176 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:42.873900 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.874249 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.874592 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.875322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:42.877335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:42.909506 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1429/2852] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:42 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:42.910884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:42.921396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b91cc8-6501-49e4-80d2-a9f196a5521e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:42.981971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:42.982613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:42.982890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:42.983163 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:42.983817 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:42.985811 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:43.008472 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1424/2853] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:42 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:43.012782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:43.024019 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce0dfc-a2f0-4ad2-a233-aa1dceb05f41 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:43.029539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:43.030152 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:43.030432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:43.030705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:43.031353 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:43.033924 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:43.046933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1430/2854] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:43.051704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:43.062214 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3763d-397e-4902-b94a-46e8024efa0e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:43.074019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84865b-2027-4662-8744-1bd7bb065394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:43.074019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84865b-2027-4662-8744-1bd7bb065394 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:43.074019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84865b-2027-4662-8744-1bd7bb065394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:43.074019 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84865b-2027-4662-8744-1bd7bb065394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:43.074433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84865b-2027-4662-8744-1bd7bb065394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:43.074433 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84865b-2027-4662-8744-1bd7bb065394 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:43.084294 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1425/2855] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:19:43.088376 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:43.100118 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e28b3e-28db-49b7-9f5b-94533dd6f7de admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:43.105548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:43.106314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:43.106598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:43.106873 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] PATH_INFO: `/v3/projects/7cf24d0de2464885ba6eac3a0193317d/users/4866263eb5fd4269adc3a2e3ca611fb4/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:43.123707 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf24d0de2464885ba6eac3a0193317d, user_id=4866263eb5fd4269adc3a2e3ca611fb4, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:43.126549 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:43.139941 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1431/2856] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:43 2022] PUT /identity/v3/projects/7cf24d0de2464885ba6eac3a0193317d/users/4866263eb5fd4269adc3a2e3ca611fb4/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:43.151132 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:43.151421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:43.151693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:43.180090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] MFA Rules not processed for user `4866263eb5fd4269adc3a2e3ca611fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:43.181223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:43.218710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:43.245580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1426/2857] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:43.257431 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:43.259125 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:43.314126 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ba1ab-28d8-4f3f-8f3e-80433940f368 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:43.318354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:43.318943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:43.319220 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:43.319489 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:43.327951 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:43.329355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:43.337959 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1432/2858] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:46.109096 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:46.110527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:46.204697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877331e-66c0-4d02-9bdc-91376cbf9933 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:46.210167 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d2edc-04fb-48d1-9d39-d84eb1c5413e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:46.210778 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2edc-04fb-48d1-9d39-d84eb1c5413e service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:46.211062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2edc-04fb-48d1-9d39-d84eb1c5413e service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:46.211336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d2edc-04fb-48d1-9d39-d84eb1c5413e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:46.213412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ca2d2edc-04fb-48d1-9d39-d84eb1c5413e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:46.225532 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d2edc-04fb-48d1-9d39-d84eb1c5413e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:46.227110 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d2edc-04fb-48d1-9d39-d84eb1c5413e service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:46.237564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1427/2859] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:53.334004 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:54.363194 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:54.364744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:54.429166 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd68a-afd1-4e0f-995f-7992403e42ab service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:54.433690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:54.434248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:54.434524 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:54.434795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] PATH_INFO: `/v3/users/df622011fc604f37923fe02f0960ea4d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:54.435930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df622011fc604f37923fe02f0960ea4d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:54.437812 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:54.483032 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df622011fc604f37923fe02f0960ea4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:54.492700 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df622011fc604f37923fe02f0960ea4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:54.505615 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df622011fc604f37923fe02f0960ea4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:54.506276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df622011fc604f37923fe02f0960ea4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:54.513089 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df622011fc604f37923fe02f0960ea4d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:54.518964 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1433/2860] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:54 2022] DELETE /identity/v3/users/df622011fc604f37923fe02f0960ea4d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:54.523451 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:54.523937 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:54.524067 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:54.542564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:54.544053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:54.601649 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1428/2861] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:54.611432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:54.613374 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:54.691468 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23734b27-049c-4040-8db5-fc015f88a56a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:54.695018 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:54.695619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:54.695926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:54.696197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:54.705104 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:54.706511 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:54.715664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1434/2862] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:55.202866 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:55.204377 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:55.217608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9648211-0248-417b-8a26-8067a50dc67b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:55.223136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:55.223892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:55.224240 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:55.224581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] PATH_INFO: `/v3/projects/55e513e86e3742aa9e9f77597f3599c6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:55.226380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e513e86e3742aa9e9f77597f3599c6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:55.228316 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:55.270635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:55.275190 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e513e86e3742aa9e9f77597f3599c6', 'request_id': 'req-4f3f1a11-52ec-43a9-9b54-6c381c924c24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:55.280869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e513e86e3742aa9e9f77597f3599c6', 'request_id': 'req-4f3f1a11-52ec-43a9-9b54-6c381c924c24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:55.287876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e513e86e3742aa9e9f77597f3599c6', 'request_id': 'req-4f3f1a11-52ec-43a9-9b54-6c381c924c24'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:55.298941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] The token cache is being invalidate because project 55e513e86e3742aa9e9f77597f3599c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:55.299233 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:55.299941 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f3f1a11-52ec-43a9-9b54-6c381c924c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:55.301394 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1429/2863] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:55 2022] DELETE /identity/v3/projects/55e513e86e3742aa9e9f77597f3599c6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:58.585403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:58.586631 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:58.594304 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:58.600690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b628be-3987-47e1-8d37-ae812dde9144 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:58.658619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:58.659344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:58.659679 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:58.659995 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] PATH_INFO: `/v3/users/8c20872adb144cdcab29f6b925a52a8f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:58.661132 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c20872adb144cdcab29f6b925a52a8f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:58.663022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:58.708422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c20872adb144cdcab29f6b925a52a8f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:58.715267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c20872adb144cdcab29f6b925a52a8f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:58.726942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c20872adb144cdcab29f6b925a52a8f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:58.727584 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c20872adb144cdcab29f6b925a52a8f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:58.734774 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c20872adb144cdcab29f6b925a52a8f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:58.740548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1435/2864] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:19:58 2022] DELETE /identity/v3/users/8c20872adb144cdcab29f6b925a52a8f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:58.744924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:58.745358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:58.745466 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:58.764329 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:58.765508 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:58.822791 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1430/2865] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:58.832439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:58.833760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:58.891103 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ad0ff-3631-40d1-bbe2-4cd122933934 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:58.894744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b33aa-a996-44f6-9ffe-fbeb666bdb7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:58.895343 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b33aa-a996-44f6-9ffe-fbeb666bdb7d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:58.895616 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b33aa-a996-44f6-9ffe-fbeb666bdb7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:58.895917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b33aa-a996-44f6-9ffe-fbeb666bdb7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:58.904557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b33aa-a996-44f6-9ffe-fbeb666bdb7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:58.905961 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b33aa-a996-44f6-9ffe-fbeb666bdb7d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:58.915439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1436/2866] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:19:59.177318 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.179136 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:59.191505 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11798bc-70ba-400c-b87b-f2e4325d6d1b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.196207 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.196834 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.197108 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.197376 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] PATH_INFO: `/v3/projects/1dcf2fd5ba1e4628b7c77c5ac5f28a54` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.198507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dcf2fd5ba1e4628b7c77c5ac5f28a54)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.200476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.245751 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dcf2fd5ba1e4628b7c77c5ac5f28a54', 'request_id': 'req-2017efc1-c5a4-4165-b522-19e67fa128e4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:59.251046 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dcf2fd5ba1e4628b7c77c5ac5f28a54', 'request_id': 'req-2017efc1-c5a4-4165-b522-19e67fa128e4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:59.258693 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dcf2fd5ba1e4628b7c77c5ac5f28a54', 'request_id': 'req-2017efc1-c5a4-4165-b522-19e67fa128e4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:59.269518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] The token cache is being invalidate because project 1dcf2fd5ba1e4628b7c77c5ac5f28a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:19:59.269801 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:59.270476 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2017efc1-c5a4-4165-b522-19e67fa128e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:19:59.272066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1431/2867] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:19:59 2022] DELETE /identity/v3/projects/1dcf2fd5ba1e4628b7c77c5ac5f28a54 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:59.277040 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c770e-baaa-4273-bbf0-f8390fbf97c0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.277326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c770e-baaa-4273-bbf0-f8390fbf97c0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.277643 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c770e-baaa-4273-bbf0-f8390fbf97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.299480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6c5c770e-baaa-4273-bbf0-f8390fbf97c0 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:59.301305 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6c5c770e-baaa-4273-bbf0-f8390fbf97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:59.319430 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:59.360973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1437/2868] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:59.370348 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be264d-6dd0-4600-8e6d-10f449650c6c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.370636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be264d-6dd0-4600-8e6d-10f449650c6c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.370911 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be264d-6dd0-4600-8e6d-10f449650c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.390135 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d9be264d-6dd0-4600-8e6d-10f449650c6c None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:59.391358 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d9be264d-6dd0-4600-8e6d-10f449650c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:59.399047 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1432/2869] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:59.403207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c770e-baaa-4273-bbf0-f8390fbf97c0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.413481 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c770e-baaa-4273-bbf0-f8390fbf97c0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.418326 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.418921 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.419201 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.419475 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.420150 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:19:59.420453 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.422037 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.437322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1438/2870] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:19:59.438429 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be264d-6dd0-4600-8e6d-10f449650c6c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.448077 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be264d-6dd0-4600-8e6d-10f449650c6c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.452964 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.453563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.453837 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.454104 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.454724 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.456748 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.488809 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1433/2871] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:19:59 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:59.490196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.503521 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51e119-fd05-4d4f-83b8-64fbdf922e53 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.559116 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.559713 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.560022 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.560296 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.560914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.562794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.585720 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1439/2872] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:19:59 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:19:59.589557 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.600666 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a681bf-6cbd-4bd2-84af-6c91334f54b3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.605912 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.606507 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.606784 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.607054 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.607705 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.609332 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.619341 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1434/2873] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:59.623599 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.633422 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cfef0-6745-4fe5-8165-691af5efb3a8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.638423 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b5f10-edc1-4bf9-942a-4a9d69eb4ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.639023 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5f10-edc1-4bf9-942a-4a9d69eb4ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.639315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5f10-edc1-4bf9-942a-4a9d69eb4ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.639572 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5f10-edc1-4bf9-942a-4a9d69eb4ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.640314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b5f10-edc1-4bf9-942a-4a9d69eb4ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.642142 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b5f10-edc1-4bf9-942a-4a9d69eb4ae7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.653744 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1440/2874] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 01:19:59.657676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.669095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977b5ae-b719-4fbc-b360-89222e05a4e0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.673642 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.674284 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.674562 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.675324 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] PATH_INFO: `/v3/projects/dfa07fc2740842a98113dcae8f46cc7c/users/258826592903462eb4a9274e96589bc9/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.687015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:19:59.690234 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa07fc2740842a98113dcae8f46cc7c, user_id=258826592903462eb4a9274e96589bc9, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.693870 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.706884 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1435/2875] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:19:59 2022] PUT /identity/v3/projects/dfa07fc2740842a98113dcae8f46cc7c/users/258826592903462eb4a9274e96589bc9/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:19:59.711775 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315aac-7a23-48a1-8448-54933989595d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.712076 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315aac-7a23-48a1-8448-54933989595d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.712347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96315aac-7a23-48a1-8448-54933989595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.730705 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-96315aac-7a23-48a1-8448-54933989595d None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:19:59.731934 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96315aac-7a23-48a1-8448-54933989595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:59.792029 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1441/2876] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:19:59.802165 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:19:59.803966 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:19:59.860613 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a3289-3ce9-4549-b907-a9105f8dd196 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:19:59.864726 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f678f-0b9c-4b44-ba5a-918db57fa521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:19:59.865323 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f678f-0b9c-4b44-ba5a-918db57fa521 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:19:59.865597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f678f-0b9c-4b44-ba5a-918db57fa521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:19:59.865864 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f678f-0b9c-4b44-ba5a-918db57fa521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:19:59.874589 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f678f-0b9c-4b44-ba5a-918db57fa521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:19:59.875971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f678f-0b9c-4b44-ba5a-918db57fa521 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:19:59.885735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1436/2877] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:20:01.692948 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96315aac-7a23-48a1-8448-54933989595d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:01.694406 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96315aac-7a23-48a1-8448-54933989595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:01.707371 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96315aac-7a23-48a1-8448-54933989595d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:01.712086 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:01.712726 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:01.713003 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:01.713291 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] PATH_INFO: `/v3/users/4866263eb5fd4269adc3a2e3ca611fb4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:01.714437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4866263eb5fd4269adc3a2e3ca611fb4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:01.716391 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:01.746131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:20:01.766854 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4866263eb5fd4269adc3a2e3ca611fb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:01.775172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4866263eb5fd4269adc3a2e3ca611fb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:01.788110 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4866263eb5fd4269adc3a2e3ca611fb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:01.788706 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4866263eb5fd4269adc3a2e3ca611fb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:01.795986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4866263eb5fd4269adc3a2e3ca611fb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:01.805821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1442/2878] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:20:01 2022] DELETE /identity/v3/users/4866263eb5fd4269adc3a2e3ca611fb4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 01:20:01.807381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:01.808082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:01.808082 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:01.827607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:20:01.828845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:01.899899 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1437/2879] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:20:01.909820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:01.913196 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:01.980520 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f02c4e-2dc0-40cd-b182-ad911f1b910e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:01.984403 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bd63e-448d-4fc2-bd42-f92f12b3f732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:01.984933 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bd63e-448d-4fc2-bd42-f92f12b3f732 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:01.985210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bd63e-448d-4fc2-bd42-f92f12b3f732 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:01.985479 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bd63e-448d-4fc2-bd42-f92f12b3f732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:01.996413 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bd63e-448d-4fc2-bd42-f92f12b3f732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:01.997899 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bd63e-448d-4fc2-bd42-f92f12b3f732 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:02.009911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1443/2880] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:20:02.489869 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:02.491865 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:02.516402 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b710fa5-891d-478c-97c1-ed95ce27da28 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:02.536518 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:02.537215 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:02.537498 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:02.537773 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] PATH_INFO: `/v3/projects/7cf24d0de2464885ba6eac3a0193317d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:02.541118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf24d0de2464885ba6eac3a0193317d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:02.544103 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:02.638062 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf24d0de2464885ba6eac3a0193317d', 'request_id': 'req-643c0034-e120-4055-940d-608ed20dffa3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:02.647658 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf24d0de2464885ba6eac3a0193317d', 'request_id': 'req-643c0034-e120-4055-940d-608ed20dffa3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:02.663020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf24d0de2464885ba6eac3a0193317d', 'request_id': 'req-643c0034-e120-4055-940d-608ed20dffa3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:02.679426 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] The token cache is being invalidate because project 7cf24d0de2464885ba6eac3a0193317d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:20:02.679805 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:02.680872 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:20:02.682424 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1438/2881] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:20:02 2022] DELETE /identity/v3/projects/7cf24d0de2464885ba6eac3a0193317d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:20:04.079865 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006b67-e72d-4aaf-9c16-20ef1de24fca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:04.080402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006b67-e72d-4aaf-9c16-20ef1de24fca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:04.080402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006b67-e72d-4aaf-9c16-20ef1de24fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:04.115558 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fd006b67-e72d-4aaf-9c16-20ef1de24fca None None] MFA Rules not processed for user `258826592903462eb4a9274e96589bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:20:04.117081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fd006b67-e72d-4aaf-9c16-20ef1de24fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:04.189877 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1444/2882] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:20:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:20:04.194927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:04.196494 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:04.216701 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:20:04.265004 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c0034-e120-4055-940d-608ed20dffa3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:04.269749 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:04.270365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:04.270648 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:04.270919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:04.271587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:04.273533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:04.302722 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1439/2883] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:20:04 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:20:04.306945 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd006b67-e72d-4aaf-9c16-20ef1de24fca None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:04.318846 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd006b67-e72d-4aaf-9c16-20ef1de24fca None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:04.352160 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:20:04.381184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:04.381803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:04.382082 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:04.382354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:04.382986 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:04.385881 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:04.415926 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1445/2884] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:20:04 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:20:04.418945 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:04.430633 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a90e5c-a69e-43ac-85c9-33934293e96c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:04.435444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:04.436125 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:04.436412 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:04.436682 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:04.437321 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:04.438939 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:04.449413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1440/2885] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:20:04.453665 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:04.463680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bf8bf-ef8e-4a2b-9e59-397bc29acc0c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:04.468254 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:04.468896 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:04.469172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:04.469442 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] PATH_INFO: `/v3/projects/68158705e62140b78cdd0a0eb963196c/users/8927c1e136bd4e3faa7cd8c7dad98dfb/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:04.485333 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68158705e62140b78cdd0a0eb963196c, user_id=8927c1e136bd4e3faa7cd8c7dad98dfb, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:04.488632 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:04.500883 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1446/2886] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:20:04 2022] PUT /identity/v3/projects/68158705e62140b78cdd0a0eb963196c/users/8927c1e136bd4e3faa7cd8c7dad98dfb/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:20:04.504852 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:04.514746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a1473-fdea-47cb-bb90-4e7e91e2fcc6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:04.569598 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40602120-8d1e-4228-a88b-840e70af66ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:04.570197 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602120-8d1e-4228-a88b-840e70af66ac admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:04.570475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602120-8d1e-4228-a88b-840e70af66ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:04.570743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602120-8d1e-4228-a88b-840e70af66ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:04.571366 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40602120-8d1e-4228-a88b-840e70af66ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:04.572985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40602120-8d1e-4228-a88b-840e70af66ac admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:04.583660 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1441/2887] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:20:04.587687 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:04.598412 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0cd91-60da-4f47-9327-01659d3fe8d1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:04.602958 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:04.603596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:04.603913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:04.604186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] PATH_INFO: `/v3/projects/68158705e62140b78cdd0a0eb963196c/users/8927c1e136bd4e3faa7cd8c7dad98dfb/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:04.606095 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68158705e62140b78cdd0a0eb963196c, user_id=8927c1e136bd4e3faa7cd8c7dad98dfb, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:04.608751 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:04.619228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1447/2888] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:20:04 2022] PUT /identity/v3/projects/68158705e62140b78cdd0a0eb963196c/users/8927c1e136bd4e3faa7cd8c7dad98dfb/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:20:09.027828 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7669d-b8f9-450e-914f-5b5a55d1b015 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:09.028297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7669d-b8f9-450e-914f-5b5a55d1b015 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:09.028297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7669d-b8f9-450e-914f-5b5a55d1b015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:09.062021 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dba7669d-b8f9-450e-914f-5b5a55d1b015 None None] MFA Rules not processed for user `8927c1e136bd4e3faa7cd8c7dad98dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:20:09.063935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dba7669d-b8f9-450e-914f-5b5a55d1b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:09.183769 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1442/2889] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:20:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:20:09.203837 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:09.206943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:09.346460 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ca306-9975-496b-a458-e5b4dda183a9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:09.352119 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:09.352836 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:09.353177 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:09.353454 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:09.418355 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:09.419993 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:09.423723 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:20:09.432308 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1448/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:20:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:20:10.456468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bb512-41c1-40eb-afdb-4a8bbcf1f86b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:10.456986 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bb512-41c1-40eb-afdb-4a8bbcf1f86b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:10.457133 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bb512-41c1-40eb-afdb-4a8bbcf1f86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:10.476644 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bd4bb512-41c1-40eb-afdb-4a8bbcf1f86b None None] MFA Rules not processed for user `258826592903462eb4a9274e96589bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:20:10.477907 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bd4bb512-41c1-40eb-afdb-4a8bbcf1f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:10.485150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1443/2891] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:20:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:20:19.432782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:20:20.470491 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:20:40.856180 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:20:40.857667 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:20:40.914467 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5b470-39b1-4521-8260-e470f6fe1b1c service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:20:40.918127 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:20:40.918733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:20:40.919017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:20:40.919289 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:20:40.928548 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:20:40.929968 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:20:40.939322 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1449/2892] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:20:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:20:50.935085 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=26 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:22:37.704717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bb512-41c1-40eb-afdb-4a8bbcf1f86b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:22:37.714469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bb512-41c1-40eb-afdb-4a8bbcf1f86b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:22:37.717998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:22:37.718592 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:22:37.719415 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:22:37.719697 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:22:37.721455 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:37.734599 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:22:37.736053 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:22:37.746908 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1444/2893] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:22:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:22:39.153733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:22:39.162734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ee2e9-1134-48b8-88b7-f16e48ff514f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:22:39.166557 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:22:39.167138 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:22:39.167414 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:22:39.167712 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:22:39.169446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:39.182979 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:22:39.184417 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:22:39.195361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1450/2894] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:22:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:22:41.297015 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:22:41.298588 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:41.309183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e8afc-944f-4668-9182-0f18c0154823 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:22:41.313365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:22:41.313957 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:22:41.314232 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:22:41.314499 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:22:41.316183 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:41.325583 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:22:41.327035 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:22:41.335659 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1445/2895] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:22:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:22:49.190359 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:22:51.094808 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:22:51.103316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc664123-25c2-4d55-a36b-6481c28c208f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:22:51.106886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:22:51.107473 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:22:51.107764 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:22:51.108041 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:22:51.109692 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:51.119483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:22:51.120915 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:22:51.128822 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1451/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:22:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:22:51.332075 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=12 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:22:52.395614 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:22:52.397134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:52.468970 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702bc2f-7bbe-4cb5-8662-adaf103d4b73 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:22:52.474974 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:22:52.475150 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:22:52.475428 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:22:52.475696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:22:52.477993 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:52.490806 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:22:52.492421 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:22:52.502312 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1446/2897] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:22:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:22:53.742510 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:22:53.745210 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:53.763316 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a74f99-e7bd-4010-8ce5-799b8552cc23 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:22:53.769222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:22:53.769930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:22:53.770311 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:22:53.770823 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:22:53.772664 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:22:53.785505 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:22:53.787084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:22:53.798504 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1452/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:22:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:23:02.498092 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=32 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:23:03.793402 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=12 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:24:20.986114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:21.002217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeb008-3acf-4db0-8758-980439e219ef service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:21.008210 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:21.009227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:21.009735 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:21.010267 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:21.012488 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:21.026237 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:21.027948 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:21.040073 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1447/2899] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:24:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:24:23.448398 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:23.462279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75243f1-cb98-48ea-8247-868a7ebb066e service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:23.469062 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:23.469661 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:23.469944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:23.470279 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:23.472038 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:23.488625 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:23.490204 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:23.504947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1453/2900] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:24:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:24:27.761847 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:27.773587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f196ddb-95bd-497a-b597-885909a3807c service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:27.778861 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:27.780095 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:27.780436 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:27.780794 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:27.782564 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:27.794500 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:27.796835 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:27.807277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1448/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:24:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:24:30.647257 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:30.659790 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b7178-c832-44d8-9fce-45b613eeabcb service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:30.681200 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:30.681887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:30.682174 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:30.682455 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:30.684576 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:30.700930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:30.703596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:30.717660 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1454/2902] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:24:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:24:30.723427 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:30.789636 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f16aa-902d-4ef7-823d-c02fe8a4bcc0 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:30.794085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:30.794704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:30.795006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:30.795273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] PATH_INFO: `/v3/users/258826592903462eb4a9274e96589bc9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:30.796396 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258826592903462eb4a9274e96589bc9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:30.798297 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:30.843712 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258826592903462eb4a9274e96589bc9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:30.844381 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258826592903462eb4a9274e96589bc9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:30.851179 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258826592903462eb4a9274e96589bc9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:30.862853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258826592903462eb4a9274e96589bc9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:30.863493 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258826592903462eb4a9274e96589bc9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:30.875584 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1449/2903] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:24:30 2022] DELETE /identity/v3/users/258826592903462eb4a9274e96589bc9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:24:31.058228 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:31.069953 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a282fd-0ae6-4c97-a190-ae8cf84c9afb service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:31.127834 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:31.128444 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.128810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.129081 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] PATH_INFO: `/v3/projects/dfa07fc2740842a98113dcae8f46cc7c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.130197 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa07fc2740842a98113dcae8f46cc7c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:31.132314 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:31.179529 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa07fc2740842a98113dcae8f46cc7c', 'request_id': 'req-18826411-65c5-415f-873d-f35eb4d24596'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.184930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa07fc2740842a98113dcae8f46cc7c', 'request_id': 'req-18826411-65c5-415f-873d-f35eb4d24596'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.192647 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa07fc2740842a98113dcae8f46cc7c', 'request_id': 'req-18826411-65c5-415f-873d-f35eb4d24596'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.204030 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] The token cache is being invalidate because project dfa07fc2740842a98113dcae8f46cc7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:24:31.204321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.204944 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.206404 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1455/2904] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:24:31 2022] DELETE /identity/v3/projects/dfa07fc2740842a98113dcae8f46cc7c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:24:31.210378 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:31.211810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:31.224563 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555e12d-23ba-43c2-95c0-9687a4061b96 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:31.273572 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:24:31.283854 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:31.284480 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.284761 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.285034 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] PATH_INFO: `/v3/users/8927c1e136bd4e3faa7cd8c7dad98dfb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.286155 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8927c1e136bd4e3faa7cd8c7dad98dfb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:31.288137 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:31.333291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8927c1e136bd4e3faa7cd8c7dad98dfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.333932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8927c1e136bd4e3faa7cd8c7dad98dfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.341696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8927c1e136bd4e3faa7cd8c7dad98dfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.353351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8927c1e136bd4e3faa7cd8c7dad98dfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.353983 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-da494bf6-c94b-4b10-997f-5694113f7fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8927c1e136bd4e3faa7cd8c7dad98dfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.366203 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1450/2905] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:24:31 2022] DELETE /identity/v3/users/8927c1e136bd4e3faa7cd8c7dad98dfb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:24:31.532996 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:31.534562 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:31.549680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18826411-65c5-415f-873d-f35eb4d24596 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:31.612253 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:31.612879 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.613157 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.613429 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] PATH_INFO: `/v3/projects/68158705e62140b78cdd0a0eb963196c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.614530 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68158705e62140b78cdd0a0eb963196c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:31.616445 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:31.622464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:24:31.664375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68158705e62140b78cdd0a0eb963196c', 'request_id': 'req-772a323a-16c0-44f7-baea-e10e47b3cff8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.669782 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68158705e62140b78cdd0a0eb963196c', 'request_id': 'req-772a323a-16c0-44f7-baea-e10e47b3cff8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.677108 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68158705e62140b78cdd0a0eb963196c', 'request_id': 'req-772a323a-16c0-44f7-baea-e10e47b3cff8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.688913 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] The token cache is being invalidate because project 68158705e62140b78cdd0a0eb963196c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:24:31.689199 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.689835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-772a323a-16c0-44f7-baea-e10e47b3cff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:24:31.691329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1456/2906] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:24:31 2022] DELETE /identity/v3/projects/68158705e62140b78cdd0a0eb963196c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:24:31.696185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d2ef7-a082-4682-98b1-8973d8da28e8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.696487 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d2ef7-a082-4682-98b1-8973d8da28e8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.696739 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d2ef7-a082-4682-98b1-8973d8da28e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.716596 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-af8d2ef7-a082-4682-98b1-8973d8da28e8 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:24:31.717710 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af8d2ef7-a082-4682-98b1-8973d8da28e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:31.775365 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1451/2907] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:24:31.784207 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172828-a5cf-4e3e-8222-765be7dc81e9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.784517 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172828-a5cf-4e3e-8222-765be7dc81e9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.784793 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172828-a5cf-4e3e-8222-765be7dc81e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.804292 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-02172828-a5cf-4e3e-8222-765be7dc81e9 None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:24:31.805570 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02172828-a5cf-4e3e-8222-765be7dc81e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:31.813957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1457/2908] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:24:31.818064 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d2ef7-a082-4682-98b1-8973d8da28e8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:31.828211 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d2ef7-a082-4682-98b1-8973d8da28e8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:31.832740 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:31.833336 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.833620 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.833890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.834528 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 01:24:31.834830 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:31.836462 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:31.847727 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1452/2909] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 6 01:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 01:24:31.851947 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172828-a5cf-4e3e-8222-765be7dc81e9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:31.860972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172828-a5cf-4e3e-8222-765be7dc81e9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:31.865537 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:31.866122 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.866399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.866669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.867293 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:31.869226 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:31.906446 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1458/2910] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:24:31 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:24:31.910635 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:31.920746 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c915af-01be-456b-b70a-f4bf9b52a7cd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:31.976785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:31.977302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:31.977581 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:31.977853 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:31.978475 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:31.980407 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:31.988223 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:24:32.002328 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1453/2911] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:24:31 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:24:32.006586 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:32.017248 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57aa787-518e-42be-855d-1121da0b13a5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:32.021796 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:32.022382 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:32.022656 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:32.022922 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:32.023553 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:32.025251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:32.034914 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1459/2912] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:24:32.038979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:32.049134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c56b3-6ebb-43b6-a5d1-8567dd2e7b73 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:32.053597 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac537bc-88d5-4484-8060-e1326fd1e292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:32.054180 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac537bc-88d5-4484-8060-e1326fd1e292 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:32.054456 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac537bc-88d5-4484-8060-e1326fd1e292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:32.054723 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac537bc-88d5-4484-8060-e1326fd1e292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:32.055339 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac537bc-88d5-4484-8060-e1326fd1e292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:32.056947 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac537bc-88d5-4484-8060-e1326fd1e292 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:32.067706 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1454/2913] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:24:32.071568 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:32.081251 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ae49a-3d05-43cf-ae47-3ea4ecd0c3c7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:32.085690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:32.086329 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:32.086605 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:32.086874 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] PATH_INFO: `/v3/projects/333491050689469789d7fd61b4d70cf8/users/e58ada13a75445df97d8b6d1e3ea9c50/roles/cb49e0a72a2040b698c806f4871e51a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:32.100992 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=333491050689469789d7fd61b4d70cf8, user_id=e58ada13a75445df97d8b6d1e3ea9c50, role_id=cb49e0a72a2040b698c806f4871e51a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:32.103596 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:32.116733 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1460/2914] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:24:32 2022] PUT /identity/v3/projects/333491050689469789d7fd61b4d70cf8/users/e58ada13a75445df97d8b6d1e3ea9c50/roles/cb49e0a72a2040b698c806f4871e51a5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:24:32.121626 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18c329-be30-4795-81f9-c2eebd73d5cf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:32.121910 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18c329-be30-4795-81f9-c2eebd73d5cf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:32.122184 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18c329-be30-4795-81f9-c2eebd73d5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:32.141737 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3a18c329-be30-4795-81f9-c2eebd73d5cf None None] MFA Rules not processed for user `1842912f886f4b17b51cd35bebba1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:24:32.142938 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3a18c329-be30-4795-81f9-c2eebd73d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:32.200333 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1455/2915] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:24:32.209785 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:32.211457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:32.267181 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066beb-4fe2-4904-9f1b-17215000a0ea admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:32.270658 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:32.271250 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:32.271527 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:32.271829 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:32.280401 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:32.281795 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:32.291172 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1461/2916] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:24:36.188930 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ffac2-00d7-4e32-84b4-73cd74b1cf75 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:36.189417 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ffac2-00d7-4e32-84b4-73cd74b1cf75 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:36.189533 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ffac2-00d7-4e32-84b4-73cd74b1cf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:36.222496 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-716ffac2-00d7-4e32-84b4-73cd74b1cf75 None None] MFA Rules not processed for user `e58ada13a75445df97d8b6d1e3ea9c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:24:36.223781 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-716ffac2-00d7-4e32-84b4-73cd74b1cf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:36.296734 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1456/2917] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:24:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:24:36.301449 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:36.312099 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64e3d4-5a6a-423e-aa58-9fb9a6c74102 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:36.316680 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:36.317276 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:36.317552 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:36.317821 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:36.318457 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:36.320415 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:36.351206 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1462/2918] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 6 01:24:36 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 01:24:36.352444 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ffac2-00d7-4e32-84b4-73cd74b1cf75 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:36.363580 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ffac2-00d7-4e32-84b4-73cd74b1cf75 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:36.422059 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:36.422676 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:36.422956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:36.423230 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:36.423936 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:36.425851 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:36.442389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:24:36.449200 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1457/2919] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 6 01:24:36 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 01:24:36.453364 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:36.459016 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:24:36.466131 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8a5a6-e347-4073-8ac1-03b73c598ed7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:36.472595 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:36.473205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:36.473483 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:36.473750 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:36.474387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:36.476034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:36.487810 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1463/2920] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:24:36.492020 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:36.502845 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415f3ed-c8ed-490f-aa88-4a3017d70aea admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:36.507539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:36.508227 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:36.508509 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:36.508785 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] PATH_INFO: `/v3/projects/a4fcc6ecf4ff4ce58d9f2f5e748ec3c3/users/91ecc92efed04e6badf49a597a8933f7/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:36.525078 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4fcc6ecf4ff4ce58d9f2f5e748ec3c3, user_id=91ecc92efed04e6badf49a597a8933f7, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:36.527743 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:36.542501 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1458/2921] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:24:36 2022] PUT /identity/v3/projects/a4fcc6ecf4ff4ce58d9f2f5e748ec3c3/users/91ecc92efed04e6badf49a597a8933f7/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:24:36.547315 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:36.557550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488327f-c6a2-4c0a-a2c0-c056bc072484 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:36.616026 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219535c-2394-4cde-b5a8-1bfd29020cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:36.616544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219535c-2394-4cde-b5a8-1bfd29020cac admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:36.616820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219535c-2394-4cde-b5a8-1bfd29020cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:36.617091 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219535c-2394-4cde-b5a8-1bfd29020cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:36.617724 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219535c-2394-4cde-b5a8-1bfd29020cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:36.619302 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219535c-2394-4cde-b5a8-1bfd29020cac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:36.629788 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1464/2922] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 6 01:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 01:24:36.633920 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:36.644486 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731f2d6-a7da-4ece-b30a-d6a2cb942311 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:36.652668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:36.653351 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:36.653629 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:36.654087 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] PATH_INFO: `/v3/projects/a4fcc6ecf4ff4ce58d9f2f5e748ec3c3/users/91ecc92efed04e6badf49a597a8933f7/roles/40b543bb076c4ea9a663ec990f6582b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:36.656277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4fcc6ecf4ff4ce58d9f2f5e748ec3c3, user_id=91ecc92efed04e6badf49a597a8933f7, role_id=40b543bb076c4ea9a663ec990f6582b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:36.659128 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:36.671277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1459/2923] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 6 01:24:36 2022] PUT /identity/v3/projects/a4fcc6ecf4ff4ce58d9f2f5e748ec3c3/users/91ecc92efed04e6badf49a597a8933f7/roles/40b543bb076c4ea9a663ec990f6582b6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:24:41.301186 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:41.302120 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:41.302347 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:41.334439 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] MFA Rules not processed for user `91ecc92efed04e6badf49a597a8933f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:24:41.335686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:41.424640 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1465/2924] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:24:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:24:41.438924 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:41.440867 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:41.529792 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1466cd-080f-4d26-ba86-cc969b6e9e9a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:41.535538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd099d-b3f3-4341-a025-8f8521e7182e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:41.536185 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd099d-b3f3-4341-a025-8f8521e7182e service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:41.536468 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd099d-b3f3-4341-a025-8f8521e7182e service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:41.536742 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd099d-b3f3-4341-a025-8f8521e7182e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:41.599927 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd099d-b3f3-4341-a025-8f8521e7182e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:41.601919 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd099d-b3f3-4341-a025-8f8521e7182e service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:41.611687 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1460/2925] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:24:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:24:47.484997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:24:47.486437 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:47.555370 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7867f-f1d3-4d01-aa22-d34cf95a4f5c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:24:47.560600 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:24:47.561203 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:47.561480 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:47.561749 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:47.563464 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:47.575780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:24:47.577354 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:24:47.587688 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1466/2926] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:24:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:24:51.607174 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:24:56.813066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729f69-390b-4965-a9ba-d481150d1388 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:24:56.813672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729f69-390b-4965-a9ba-d481150d1388 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:24:56.813672 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729f69-390b-4965-a9ba-d481150d1388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:24:56.832217 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-19729f69-390b-4965-a9ba-d481150d1388 None None] MFA Rules not processed for user `e58ada13a75445df97d8b6d1e3ea9c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:24:56.833956 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-19729f69-390b-4965-a9ba-d481150d1388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:24:56.840903 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1461/2927] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:24:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:24:57.583606 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:25:06.826890 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:25:13.069227 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:25:13.070614 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:25:13.127361 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d6cc8-bc08-4abd-adeb-056358186c8e service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:25:13.131404 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:25:13.132034 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:25:13.132312 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:25:13.132581 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:25:13.141519 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:25:13.143184 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:25:13.152911 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1467/2928] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:25:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:25:23.148887 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=26 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:27:13.752545 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50d1e5-3456-4b9b-b621-c9a0fe43496b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:27:13.753090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50d1e5-3456-4b9b-b621-c9a0fe43496b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:27:13.753090 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50d1e5-3456-4b9b-b621-c9a0fe43496b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:27:13.772273 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2b50d1e5-3456-4b9b-b621-c9a0fe43496b None None] MFA Rules not processed for user `e58ada13a75445df97d8b6d1e3ea9c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:27:13.773473 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2b50d1e5-3456-4b9b-b621-c9a0fe43496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:27:13.780765 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1462/2929] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:27:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 01:27:14.086514 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:27:14.095144 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d9412-513d-44d6-931a-29d8d576dc0d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:27:14.098760 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a974d-e548-42f4-92db-eb230419614c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:27:14.099341 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a974d-e548-42f4-92db-eb230419614c service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:27:14.099619 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a974d-e548-42f4-92db-eb230419614c service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:27:14.099917 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a974d-e548-42f4-92db-eb230419614c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:27:14.101590 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cc6a974d-e548-42f4-92db-eb230419614c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:27:14.110894 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a974d-e548-42f4-92db-eb230419614c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:27:14.112583 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a974d-e548-42f4-92db-eb230419614c service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:27:14.121604 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1468/2930] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:27:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:27:23.766547 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:27:24.117791 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:27:45.247106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50d1e5-3456-4b9b-b621-c9a0fe43496b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:27:45.256998 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50d1e5-3456-4b9b-b621-c9a0fe43496b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:27:45.260537 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:27:45.261130 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:27:45.261403 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:27:45.261668 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:27:45.263359 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:27:45.273106 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:27:45.274514 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:27:45.282277 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1463/2931] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:27:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:27:55.278876 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:29:06.754930 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2fdd1-21bb-47e7-80f2-a11b13e38bff None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:06.755448 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2fdd1-21bb-47e7-80f2-a11b13e38bff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:06.755571 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2fdd1-21bb-47e7-80f2-a11b13e38bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:06.774598 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4cd2fdd1-21bb-47e7-80f2-a11b13e38bff None None] MFA Rules not processed for user `e58ada13a75445df97d8b6d1e3ea9c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 01:29:06.775825 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4cd2fdd1-21bb-47e7-80f2-a11b13e38bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:06.782721 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1469/2932] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 6 01:29:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 01:29:06.825485 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:29:06.826863 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:06.901892 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759b35e-8397-4d2b-acd0-315ffb8ccd6d service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:29:06.905380 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d4ca9ea488eb0aad2b328de193c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:29:06.905971 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:06.906248 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:06.906516 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:06.908302 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:06.922391 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:29:06.923849 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:29:06.934670 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1464/2933] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:29:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:29:07.217335 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2fdd1-21bb-47e7-80f2-a11b13e38bff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:29:07.226875 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2fdd1-21bb-47e7-80f2-a11b13e38bff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:29:07.230393 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:29:07.230985 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:07.231267 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:07.231538 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:07.233274 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:07.242539 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:29:07.243981 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:29:07.251734 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1470/2934] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:29:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:29:07.690043 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:29:07.691814 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:07.706389 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a014501-3ae5-4b4f-8bbe-93143ecf3663 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:29:07.711360 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:29:07.711984 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:29:07.712118 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:07.712469 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:07.712803 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:07.714615 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:07.720835 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e420848-ef82-4c07-85a0-79d9161f622d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:29:07.725794 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:29:07.726387 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:07.726668 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:07.726943 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:07.727345 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:29:07.728820 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:07.729068 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:29:07.739824 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1465/2935] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:29:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:29:07.744542 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:29:07.745957 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:29:07.756942 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1471/2936] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:29:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 01:29:17.734767 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=41 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:29:17.752017 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=15 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:29:19.817364 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:29:19.829754 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a9c33-8d82-46e4-bcb4-fc1836faa5c4 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:29:19.836024 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:29:19.836566 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:19.836844 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:19.837111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:19.838822 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:19.851002 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:29:19.852607 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:29:19.863091 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1466/2937] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:29:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:29:23.735339 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:29:23.748261 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f1b64-3e60-4c76-bab7-93795bb88e1d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:29:23.753803 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:29:23.754525 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:23.754886 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:23.755222 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:23.757006 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:29:23.769972 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:29:23.771543 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:29:23.782569 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1472/2938] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:29:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:29:29.858745 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:29:33.778669 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:29:54.489831 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:29:54.548006 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05f0e9-16f5-4b5b-8234-0bb8bb883eb8 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:29:54.553542 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:29:54.554134 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:29:54.554413 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:29:54.554683 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:29:54.562985 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:29:54.564587 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:29:54.575066 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1467/2939] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:29:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:30:04.570731 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=27 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:30:50.917262 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:30:50.953185 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216fe4-0557-4a0a-a658-c77ac2e51db5 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:30:50.956777 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:30:50.957375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:30:50.957653 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:30:50.957925 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:30:50.966954 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:30:50.968405 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:30:50.977074 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1473/2940] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:30:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:30:53.982608 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:30:54.015465 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838fb0a-0d8b-4695-93b6-0becfccc37ae service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:30:54.021309 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b6caf1ad3784ddabda79ca0b1fc468d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:30:54.021918 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:30:54.022898 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:30:54.023175 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:30:54.024958 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:30:54.061085 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:30:54.062667 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:30:54.073894 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1468/2941] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:30:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:31:00.973049 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=18 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:31:04.007773 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:31:04.016550 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830973fe-1251-4f90-b0fd-e9be75f8a1c8 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:31:04.020100 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc47f8d295ed481a87b4fb0d405c605e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:31:04.020690 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:31:04.020970 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:31:04.021244 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:31:04.068677 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=22 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:31:04.092956 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:31:04.094375 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:31:04.113162 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1474/2942] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:31:07.357780 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:31:07.369577 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecbcac-3344-4c4e-aadb-66a21307957a service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:31:07.374917 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:31:07.375527 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:31:07.375839 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:31:07.376114 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:31:07.377832 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:31:07.406164 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:31:07.408760 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:31:07.422651 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1469/2943] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:31:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:31:10.207272 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:31:10.219971 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4e685-545a-4900-b337-4a56fde80677 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:31:10.226264 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:31:10.226831 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:31:10.227109 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:31:10.227385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:31:10.229189 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:31:10.242101 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:31:10.243686 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:31:10.256544 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1475/2944] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:31:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:31:10.273655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:31:10.297717 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e6b73-bade-4ce1-8e79-c6ff84dc6ba3 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:31:10.303574 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:31:10.304260 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:31:10.304539 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:31:10.304808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] PATH_INFO: `/v3/users/e58ada13a75445df97d8b6d1e3ea9c50` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:31:10.305940 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58ada13a75445df97d8b6d1e3ea9c50)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:31:10.307889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:31:10.354038 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58ada13a75445df97d8b6d1e3ea9c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.354704 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58ada13a75445df97d8b6d1e3ea9c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.362188 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58ada13a75445df97d8b6d1e3ea9c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.374808 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58ada13a75445df97d8b6d1e3ea9c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.375368 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58ada13a75445df97d8b6d1e3ea9c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.387291 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1470/2945] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:31:10 2022] DELETE /identity/v3/users/e58ada13a75445df97d8b6d1e3ea9c50 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:31:10.553317 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:31:10.563904 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7609a-346d-4a6c-88df-948c48727891 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:31:10.622714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:31:10.623344 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:31:10.623627 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:31:10.623939 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] PATH_INFO: `/v3/projects/333491050689469789d7fd61b4d70cf8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:31:10.625139 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=333491050689469789d7fd61b4d70cf8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:31:10.627005 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:31:10.674466 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333491050689469789d7fd61b4d70cf8', 'request_id': 'req-30b8513c-6d8b-4645-85db-62a60f3df272'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.680591 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333491050689469789d7fd61b4d70cf8', 'request_id': 'req-30b8513c-6d8b-4645-85db-62a60f3df272'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.688084 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333491050689469789d7fd61b4d70cf8', 'request_id': 'req-30b8513c-6d8b-4645-85db-62a60f3df272'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.689394 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:31:10.713780 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] The token cache is being invalidate because project 333491050689469789d7fd61b4d70cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:31:10.714073 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.714683 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.716205 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1476/2946] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:31:10 2022] DELETE /identity/v3/projects/333491050689469789d7fd61b4d70cf8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:31:10.720372 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:31:10.721758 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:31:10.734922 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fa93-b129-4b6b-94ad-9a6ff8b98153 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:31:10.790310 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:31:10.790935 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:31:10.791212 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:31:10.791481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] PATH_INFO: `/v3/users/91ecc92efed04e6badf49a597a8933f7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:31:10.792621 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ecc92efed04e6badf49a597a8933f7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:31:10.794538 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:31:10.839604 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ecc92efed04e6badf49a597a8933f7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.840285 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ecc92efed04e6badf49a597a8933f7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.842768 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:31:10.847691 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ecc92efed04e6badf49a597a8933f7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.861254 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ecc92efed04e6badf49a597a8933f7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.861889 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ecc92efed04e6badf49a597a8933f7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:10.873757 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1471/2947] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 6 01:31:10 2022] DELETE /identity/v3/users/91ecc92efed04e6badf49a597a8933f7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:31:11.053432 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:31:11.055033 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:31:11.073443 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8513c-6d8b-4645-85db-62a60f3df272 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:31:11.132973 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1842912f886f4b17b51cd35bebba1a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '921fe46fc085463aa0816bc6ba906e3c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:31:11.133685 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:31:11.134024 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:31:11.134225 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] PATH_INFO: `/v3/projects/a4fcc6ecf4ff4ce58d9f2f5e748ec3c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:31:11.135509 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4fcc6ecf4ff4ce58d9f2f5e748ec3c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:31:11.137324 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:31:11.185714 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4fcc6ecf4ff4ce58d9f2f5e748ec3c3', 'request_id': 'req-b148b3d7-80a8-4024-bad4-93736ce02e6f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:11.191385 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4fcc6ecf4ff4ce58d9f2f5e748ec3c3', 'request_id': 'req-b148b3d7-80a8-4024-bad4-93736ce02e6f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:11.200399 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4fcc6ecf4ff4ce58d9f2f5e748ec3c3', 'request_id': 'req-b148b3d7-80a8-4024-bad4-93736ce02e6f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:11.213240 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] The token cache is being invalidate because project a4fcc6ecf4ff4ce58d9f2f5e748ec3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 01:31:11.213955 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:11.214461 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b148b3d7-80a8-4024-bad4-93736ce02e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 01:31:11.215997 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1477/2948] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 6 01:31:11 2022] DELETE /identity/v3/projects/a4fcc6ecf4ff4ce58d9f2f5e748ec3c3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 01:31:20.870374 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=3,keystone:SELECT=12 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:31:21.209321 dv-cinder-847732-2-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 01:34:56.472696 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 01:34:56.474101 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:34:56.541810 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca824c3-ac89-415f-b088-d0a3a3e41a11 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 01:34:56.599478 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c7e2e-f07d-497e-946e-239f96d9240b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3628a46d8f894d5481112ad00479a639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e0c168c2f1476599951c75ac91e4e7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 01:34:56.600111 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c7e2e-f07d-497e-946e-239f96d9240b service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 01:34:56.600388 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c7e2e-f07d-497e-946e-239f96d9240b service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 01:34:56.600655 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c7e2e-f07d-497e-946e-239f96d9240b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 01:34:56.602481 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-722c7e2e-f07d-497e-946e-239f96d9240b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 01:34:56.613932 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c7e2e-f07d-497e-946e-239f96d9240b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 01:34:56.615490 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c7e2e-f07d-497e-946e-239f96d9240b service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 01:34:56.625139 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1472/2949] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 6 01:34:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 01:35:06.620979 dv-cinder-847732-2-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=58 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}